We’re happy to share that we created our very first AI demo application.
The application is built on top of the Rust library candle by Huggingface and is running a Llama2/TinyStories model in CPU with SIMD Wasm instructions.
Interestingly, in this demo, the AI model runs on the client side, in the read-only, non-metered part of the Linera application (aka the service part).
This mode of operation is great to run user AI queries off-chain (hence privately) while leveraging the security of the Linera VM and the immutability / transparency of model parameters. This mechanism is also the first step towards on-chain AI inference (within transactions) in Linera microchains.
This is just the beginning! Stay tuned for future improvements and new AI demos!
Hi Mathieu - I got some great feedback from JV that helped me come up with the interpretation below. Is there anything you’d want to add/correct?
Summary The demo app demonstrates how we can securely store and trustlessly serve model parameters to an AI using Linera and receive the AI’s response privately.
Key Points
An interactive AI application has been integrated with Linera for the first time
It will work with Linera’s future browser extension client
The parameters for the model used by the application are stored within a smart contract on a microchain (immutable and transparent) where anyone can inspect them
Linera’s validator network ensures that the correct model is stored on-chain and served correctly to users
In this case using it is free as it runs on the non-metered Linera service
Linera’s non-metered service is ideal for running user queries locally (inside the user’s browser extension) including AI queries, without incurring gas fees
This model is also the first step towards proving how AI could integrate with Linera in a use case that would lead to related on-chain transactions.
