@nosherwan I made a neural net in ruby. We trained it on fantasy football data and for a given player it would predict game statistics, running/passing yards, tackles, touchdowns etc. It was able to emergently discover that quarterbacks don't make receptions in a normal game without explicit instructions.
You can start with any language you're comfortable with. Performance constraints will mean compiled languages will outpace interpreted, But there's lots to learn before you hit those limits
@nosherwan Not at all proficient but going on a similar journey. Tools like Langchain and tensorflow etc are all available for typescript/JavaScript.. so I’m not sure that switching to Python is at all necessary anymore.
@nosherwan python has some nice libraries. I for example started by implementing a convolutional net just in numpy, but if you're not interested in that, you can go for some specific packages, like tensofrlow, that also has nice tutorials.
@nosherwan sure - I wanted to come up with a chat (turn based) trivia game to start. Also porting over the logic to Swift now so I can use a mobile version.
I would start by thinking about how AI is mostly just function approximation (neural nets, fuzzy logic) and optimization (genetic algorithms, simulated annealing).
@nosherwan putting more thought into it I think the main challenge for a homebrew LLM is the first L. The companies publishing these models are using unfathomably Large data sets and I'm not sure how a hobbyist or small business would replicate it.
After you have a large data set and a ML algorithm (which are all well documented), you just train the model until it starts producing well formed sentences. The data is the differentiator
Add comment