Holly Spirit!
Have tried out DeepSeek R1 now!
And it's so much creepy scary! 😱
Especially the speed (tokens per second) and reasoning, it reasoning, i.e. thinking our loud, like a good old Unix programmer! 😱
I've feed it some sample of code I wrote recently for enclave's init system (just about 400 SLoC in Rust), I was rewrote it from C to Rust, and asking about the ways of improvements in handling Unix signals for processes in Rust in an idiomatic POSIX way but using Rust standard library. Overall it should be the fully fledged Init system for Linux residential processes executed and existed in enclaves.
And you know what?
OpenAI GPT4o cannot solve this - always thinking for too long, making output, but not precisely to my prompts. And then asks to pay for subscription ('cause time and tokens limits exceeded). 😂 Probably OpenAI do this intentionally to be more commercially efficient. Just a wasting of time and money.
But I subscribed out of curiosity.
OpenAI GPT o1 - already with subscription. Same things. It cannot solve prompt to the fullest. Just 400 SLoC to analyze from source code and it always stops, asking for refinements and in final not giving fullest results, just code snippets, that aren't helpful, more like a hallucination (do not use any substance and make code! 😂).
LLama 3.1 70B self-hosted - works good. Not reasoning perfectly but gives meaningful hints. Downsides - also always stops, asking for refinements and cycling, this dialogue never end and you've never reach the final meaningful complex result. Code snippets with examples are helpful. Can be use as fast search engine for code samples with context for current task. Overall helpful.
DeepSeek R1:
It's mind-blowing! 😱
One precise prompt.
Precise analysis, weak and good points, snippets, examples.
Precise reasoning, thinking out loud, as me talking with my computer science teacher in University.
And speed - it's blazing fast! Tokens per second performance is way much faster than others, even visually!
By one full run I've got all the answers for my questions.
Best pair programming session with AI overall!
GitHub Co-Pilot sucks in comparison to R1!
This shows us that even in corporate monopoly market small companies can make big shifts, bring big difference and value, can innovate, and outperform giants.
My thoughts:
We're all will be replaced by such AI creatures! 😱
Joking! We can collaborate and create beautiful things! World is definitely changing now! We can adapt and adopt these technologies, and use them for the great good! (And I'm still believe in bright future.)
Overall, LLMs as neural network has inputs and outputs, and as an input for now it requires operator, engineer, human. It cannot make goal-setting via prompting itself! (At least for now!)It's and interesting case and pair programming is so good application for reasoning LLMs and SLMs!
Paper:
https://github.com/deepseek-ai/DeepSeek-R1/blob/main/DeepSeek_R1.pdfTry it by yourself:
https://chat.deepseek.comhttps://github.com/deepseek-ai/DeepSeek-R1?tab=readme-ov-file#6-how-to-run-locallyhttps://github.com/deepseek-ai/DeepSeek-V3Weights in tensorlayers HF format:
https://huggingface.co/deepseek-ai/DeepSeek-R1Inference runners:
https://github.com/vllm-project/vllmhttps://docs.vllm.ai/en/latest/serving/distributed_serving.htmlhttps://github.com/sgl-project/sglangFor comparison with accessible alternatives:
LLama 3.1 70B chat service:
https://duck.aihttps://chatgpt.comhttps://claude.aiAnd try it out to deploy by yourself LLama 3.1/3.3 70B/405B via self-hosted installation with some custom inference runner (llama.cpp for example, or its Rust bindings) or cloud deploy from HuggingFace, and compare:
https://huggingface.co/meta-llama/Llama-3.3-70B-Instructhttps://huggingface.co/meta-llama/Llama-3.1-70B-Instructhttps://huggingface.co/meta-llama/Llama-3.1-405B-Instructhttps://github.com/ggerganov/llama.cpphttps://github.com/mdrokz/rust-llama.cpphttps://github.com/utilityai/llama-cpp-rs#AI
#AGI
#LLM