New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Define Multidimensional Array Type #18
Comments
Good points. So, in order:
Thanks for pointing it out! |
|
Would be best to support an arbitrary number of dimensions using generics, a bit like what @tensorflow/tfjs-core does with a generic tensor + aliases.
Another useful Typescript helper type I use often when doing ml in js is the That lets you properly cast tuples with an arbitrary dimension, eg. export type Tuple<T, N extends number> = N extends N
? number extends N
? T[]
: _TupleOf<T, N, []>
: never;
type _TupleOf<T, N extends number, R extends unknown[]> = R["length"] extends N
? R
: _TupleOf<T, N, [T, ...R]>; Hope it's not off-topic! Pretty excited to see where this goes! Will be happy to test/bench this against |
@mgcrea That's true, Tuples are a really useful data type in ML. Thanks for pointing it out! About the arbitrary dimension number, it could also work, but for normal ML and NLP applications, I believe we would need to go to at least Rank 4. |
@mgcrea @eduardoleao052 I made a first attempt at adding those types in this PR #21 there are still a few type errors left that require some careful thought. The added |
I'll take a look and let you know, but pretty sure it's not necessary. Thanks again! |
Rather than defining your own multi-dimensional array type, one option you could consider is directly using, and building on, existing primitives already available in other libraries. E.g., |
@kgryte Thanks for the tip! Really appreciate it. I'm just initially against adding bindings and necessary dependencies, especially as the library is currently completely independent and "from scratch". However, it is still possible to draw inspiration from the implementations on |
@eduardoleao052 Fair enough. If there isn't the goal of ecosystem interoperation, then creating a custom type for use in a walled garden is certainly sensible. This stated, if interoperation is a goal, then conforming to existing data structure interfaces is easier to do at the beginning, rather than much farther down the line when significant refactorings would then be required. I should note that standardization and interoperation is the principle goal of the Consortium for Python Data API Standards, of which PyTorch is part. And there, we've intentionally standardized a minimal array type, similar to what is present in |
@kgryte that makes sense. I think interoperation could bring immense benefits to the library and its usability. I'll study in the next few days how I could do that in a way that works with the other libraries in the ecosystem! |
It seems that most of the
any
s are due to passing around arrays of various shapes. I think we can narrow down the type to either 1 or 2 dimensional arrays of numbers. Something likeMNArray = number[] | number[][]
might do. Are there any situations where arrays would be more deeply nested?I'm curious if the arrays also need to hold class instances, there are several places where there is a check like
typeof a === "object"
but it mostly looks like this is being used to check ifa
is an array? If it is a check for an array it would be better to useArray.isArray(a)
.I could make a PR for those changes if my assumptions turn out to be correct. Just let me know.
The text was updated successfully, but these errors were encountered: