Wöchentliches KI-Update: DeepSeek Harness, Model-Routing und Scaffolding-Trends

Peace. Artificial intelligence weekly updates every Friday at 2 p.m. New York time. Today is uh August 21st, Friday. And as you see, a lot of updates and epigraph for today's presentation. The frontier is shifting from larger models to better scaffolding around the models we already have. And this is a picture of harness or scaffolding. And uh let's look at our leaderboard. So well it it it doesn't change much. We still so on the left is regular text chatbot. On the right is coding. Coding is dominated by blue color which is claude. So you see claude oppus club fable. Green is uh open source. Uh so we have Kim K3 and GLM uh 5.3. Uh we also see Quen 3.8 of course and 3.7 um Muse Spark from Meta and anyway uh red is Gemini here are the the colors. So Claude is blue, Gemini is red, OpenAI is yellow and open source is uh green. Uh okay next uh cost per intelligence index uh task again so the same uh tasks are given to different models they have different prices for their tokens but they use different uh uh number of tokens. So overall uh this is the total cost of executing the these tasks. And you see on the high end we have uh Claude Fable which is very expensive like $3.14. On uh the left we have GPT56 from OpenAI. This is Luna Luna Max. It's only 5. So compare $35. Here we also have uh Muse Glimmer from Meta. We have Neatron Lightning. Uh we have Gemini 3.5. These are all Miniax M3 cloth haiku 22 cents and so on. You you see the models [snorts] and way where is the cheapest DeepS deepseek V4 Pro? They increased the prices. So you see they used to be here in about 5 cents. Now they're like 25 cents. they still much cheaper like 10 times cheaper than American frontier models and next uh model routing strong trend. So what I'm talking about uh companies using multiple models and they uh route between models. So for example typical task you use expensive good model for planning and then you use less expensive for let's say coding and then you use again good model for reviewing right and uh well u everybody is doing this this is becoming standard now stripe buys open router for over $7 billion I even seen somewhere $8 billion so they say 88% of Forbes AI 50 including OpenAI and Antropic are building on its platform on Stripe. So this this is actually interesting. So Stripe has a service called Atlas uh to help companies to form companies mostly in Delaware. So let's say you want to create a startup. You need to register the company. So it has a mailing address and everything officially. Then you give companies stripe service to collect money from customers and also most of the startups now using AI. So you can provide them with open router to provide them AI in a most cost-ffective manner. So stripe becomes this um one place where all startups want to go. [snorts] So they want to manage not just money uh but also AI. Interesting. Codex via Omni route. So what Omni route does, it allows you to switch uh between different providers. So providers of AI services usually have a free tier, but it has uh limitations, right? But let's say you're using 10 different providers and you use a free tier from one provider and as soon as this free uh expires, you immediately switch to another model so that the user doesn't even notice. So you can uh this way basically use uh models for free. So the the project is Omni Omni Route and here it is on GitHub. Uh yes. Oh, AI singularity has begun. This is a phrase from actually from Stripe. They were explaining why they spent money on uh open router. Okay. Quen 3.827B. It becoming very uh popular. It can run on your computer and it's it's a great model. It's a local model, but it's it's a great model. Uh so it's from Alibaba open weight Apache license. So it's uh uh open dense multimodel so text and images and relatively long context for a small model only 27 billion parameters it's a very small model and if you use quantized version for 4bit it you can run it on a regular video card with 24 um GBTE uh strong agent coding UI generation uh three-dimensional demonstrations gamesing It is unusually capable for a dense 27B model. Uh this is from some publication uh built on quen uh architectural foundation. It has MTP which is multi-token prediction. So MTP head drafts multiple tokens and then the full model verifies uh them and kind of synthesize them. You you need to enable it though like in llama CPP you have to provide an option to use draft MTP functionality. Uh okay. Uh next JLM 53 uh made a version which is uh for cyber security and it actually in some tests it beats the legendary MIUS. Remember in April, Antropic released the MAS model and it was very closed only for military only for government uh because uh they were scared uh of its ability. Uh anyway, cyber security you see 84.5% versus 83.8 on cyber gym. Uh so yes, very interesting and this is GLM53 just a reminder this is open model from China. uh VA vision language action. So we're talking about robots. So this is a picture of the robot vision language action. So VA models let robots combine camera observations, natural language instructions and robot state to generate control actions. They aim to replace brittle task specific pipelines. There's a library called open VLA. Uh here you see this is a GitHub uh major open implementation uh small model 70 bill seven sorry billion parameter model trained on almost uh million robot episodes from open x embodiment data set. This is a separate area and by the way the China is of course uh ahead of us by like 100 times. uh typical deployments retain deterministic low-level control. Uh current research focuses on improving data efficiency and and and so on. Anyway, so trend shift from classic VA which is vision language action towards world action models WS which model future video world dynamics alongside action selection. Nvidia framed this Cosmos 3 as an open model and data set collection and hosted a discussion on VA W which is again world action models and hybrids. Uh Diner Robotics announced Dina 2 a W pretined on more than 1 million hours of egocentric human video research continues on val memory evaluations and so on. So this is a very very hot area. I usually don't talk about it, but somebody asked me, so I made a slide about it. Okay, this is about my channel. The channel name is Left Selector, which is my name. Currently, I have more than 7,000 subscribers, more than 300 videos. Post videos every week. Uh slides are provided on GitHub and u Google Drive. Uh look under the videos. And if you have any questions or comments or feedback uh please please pause the video and write your comments. Claw work in Chrome. So Antropic uh Chrome side panel now runs uh a full CLA session rather than isolated browser chat. Uh so this is only for Max and team uh subscriptions. the pro will uh rolling out faster uh I mean later but yeah this is great so now you are getting not just a chatbot but actually agent working in the browser entropic multi- aent research shows that agents with conflicting undisclosed goals can interpret ordinary code changes as hostile inference in interference in a shared migration task agent escalated into a turf war disabling accounts, killing rival processes, hiding malicious code and so on. Yeah, this is uh well not practical for us but it's really interesting. So strogger agents may negotiate truses more effectively but capability alone doesn't guarantee cooperation. Anyway, uh clo skills lazy context loading. So when you working with cloud code and you're using multiple skills uh there are project uh scope skills. So you put them inside your project in dotcl directory and then skills subdirectory and then for each skill you can uh create directory with this name where you put the skill file and some other files which you need for this skill. You can also similarly create global skills on your computer for all your projects. This is like your your personal stuff. You put it in your home directory underclude uh directory. So clude initially sees only the skills name and description. So each skill is a text file. It's markdown file and it starts with this short uh like description and the claude will load it and then it will load full instructions only if this skill is needed. Okay. Keep each skill focused. Store lengthy example scripts templates in in separate files. Uh set disable model invocation true for deployment or destructive workflows that should run only when explicitly requested. Okay. Uh, OpenAI GPT56 Soul ultraast on Cerebras. So, you know, Cerebbras um, it's a company uh, creating chips which like 8 and 1/2 in in size uh, and uh, they are very very fast, very very big and very fast. So uh GPT soul which is the biggest model of open AAI running on those gives uh 750 tokens per second which is 14 times uh faster than standard on let's say Nvidia. Uh OpenAI also introduced opt-in computer history intended to help Judge GPT suggest automations from selected app activity. Okay. Eged. So HD is a startup and they raised 700 million at 21 billion valuation which by the way just a month ago it was only 10 billion. So it grows very very fast and it what it does it designs specialized inference chips or systems for uh transformer models faster more cheaply than general purpose GPUs company reports over 1 billion in uh customer contracts and so on. It's a custom chips uh Google Sheets canvas uh boards mini apps. So what we're talking about uh if there are multiple systems like for example PowerBI or whatever where you have table data and then you want to create dashboards right so uh here we have a Google sheets which is like spreadsheet like table and you want to create dashboards and the these are uh examples of those uh dashboards and you can create it in what's called Google Sheets canvas It uses Gemini to convert spreadsheet data into interactive promptu mini applications like dashboards, convent boards, calendars and and so on. It is live read and write layer. So this is very important. This application it shows data and you can change data here and it will change it in the underlying uh Google sheet. Uh it is a live readrite layer over a sheet. Uh create in Google Sheets via tools. Insert bottom. Encoding is required but source data should be organized in a single tab. Okay. So this is great tool. Really great. Uh deepse harness. Uh yes this is quite amazing. This is actually I looked yesterday it was 173,000 stars. Today it's already 180,000 stars. So in just one week the deep sea harness jumped into almost 200,000 stars on GitHub. It's a absolute record. I haven't seen any GitHub uh project growing that fast. So what they have done? So they launched the latest version of their model and they also launched open-source harness which you can use with this model but not only with this model with uh any model and they made it modular and open source. Uh modular means uh this is their philosophy uh everything is a plug-in. So models, tools, sandboxes, storage, agent loops, scheduling, everything uh is pluggable. So you can substitute, you can improve, you can make it better. Uh it includes configurable agent presets and official integrations for cloth code and codex. Uh they say that it's very very raw treated as as experimental. But there are multiple videos on YouTube which are comparing let's say running uh clo code harness versus dipssec harness or uh running codex versus dips uh whatever and it's it's a good harness uh open source clo code is a great harness it existed for about year and a half I think it was introduced in February of 25 so it's a year and a half millions of people used it a lot of experience a lot of improvements, a lot of money put into it. But now we have this but it's closed code. Deepc harness is open code. That means that community will start improving it. And we should see very very fast progress with that. It's already happening and it's very easy to install it like clot code. You just install it with npx and uh and run it. There is no stable Python SDK yet. Uh workflows JSON versus Python code. So this is a very interesting article it's called the beta lesson of tool calling you seen on on archive and uh what they were trying okay the the first models like GPT3 it was just predicting next token so you ask a question in text and it replies you in text then later uh people started uh requiring that the model returns you structured data for example JSON for example you want to call a function. Let's say you have 10 different functions and each function generates a different financial report and you tell it generate a report like uh here I have an example P&L which is profit and loss report for particular company or department for specific time window. you tell it in language and what model does it converts it into a structural like JSON which tells uh what's the name of the function to call and what parameters to give it and this worked like about 2 years ago this was the standard how you do function calling the model returns JSON but if you recently worked with uh I don't know Gemini GPT clo anthropic you've seen what they do sometimes uh you ask them to do something and they generate Python code on the fly uh to do it. So the models now are very good at generating code and they fly it uh and they run it as part of the execution the answering your question. So what uh researchers here done is say okay we can ask the model to return JSON or we can ask the model to return Python script and see which one works better and what they found is if you're using the modern models recent models and especially for more complicated tasks where you have a wide selection of tools let's say for example more than 26 fun different functions you have deep chains, you have large fan out, uh it's better to ask the model to give you a Python script. So you tell it explicitly just write write code, write uh script. But if you're dealing with the old models like GPT4 or GPT4, then they don't know how to write scripts effectively. So they better perform with uh JSON. So you can do either way but looking forward it looks like it's better to ask model to write code. Uh so let's say consider the workflow how you store a workflow. You can store it as a JSON. So it's a structure which describes all the nodes and uh uh all the functionality of the nodes and the connections. It's just a JSON structure. But you can also describe it in in code in Python. And this is exactly what leng graph uh library does. So this is example of leng graph code right it uh creates a builder then it add node add age at age and then it compiles. So the structure of the workflow is stored as a python object. It's stored as code. It's built as code. Okay. Uh next, OpenAI chat GPT work. This is for paid subscriptions. This is actually screenshot from my screen. I have a paid subscription. Uh you see I have chat and I have work. Um other providers have uh chat or computer or something else. So now everybody uh have this dual approach. They have regular chat and they have work and work does actual work. It delivers the result. So it's identic mode for outcome based tasks works in cloud. You need to provide description of what you want to do. Choose the right task or plugin. Let it gather context plan steps. Use approval tools and so on. Okay. So now uh agent becomes part of a GPT uh interface on on the cloud. Uh next AI agent cyber security risks. Uh well, as models become smarter and clever, they increasingly capable of uh doing harmful stuff. Uh in testing, open AI linked agents reportedly escaped the restricted environment. Similar evaluations were done in other uh companies in including Antropic, for example. Capable agents can discover vulnerabilities, exploit systems, use deception when safeguards or containment fail. Uh so what everybody all the companies should think about is stricter containments, independent testing, incident disclosure, stronger provider liability and training models to avoid uh unacceptable paths to go. Anyway, this this is a big problem. Everybody is aware of this. Uh next uh auto design evolving AI hardness. So as hardness now becomes this frontier where companies compete including Deepseek hardness um what if uh you can actually use evolutionary approach and allow harness to improve itself. So don't train the LLM evolve the harness instead. So this is this is the approach you don't change LLM but you change the hardness you improve uh the hardness. So this is a paper you see archive paper. So uh auto designs scored uh higher exceeding claw design uh and uh the core lessons specialized self-improving hardness or scaffolding can substantially improve fixed models without fine-tuning their weights. So now the focus is shifting from creating better models to creating better harnesses or improving harnesses over time. So when people talk about hardness, they talk about loop engineering and hardness engineering in general. So loop engineering built goal-driven agents that repeatedly reason, act, observe, evaluate and revise until condition satisfied or or stop. Right? So this is uh looping. Hardness engineering is the production layer around agents context and memory management MCP tool access model context protocol access model routing guardrails tracing and observability. So you see a lot of functionality when you're talking about hardness like for example when clot code uh leaked uh people found it's about half a million lines of code. It's not a simple thing. Uh okay SQL light for small web applications uh there are two databases which are very very common one is SQL light uh which is basically uh like a from Python it's a module import SQL light and there's posgressql which which is which is a great database right so SQL light saves the whole database in one file uh several connections can read from this file at the same time in parallel but only one connection can write into it uh which uh means that when it's writing other connections should wait right whereas in posgressql uh there is no such limitation if uh like two connections can write in the same table but in different rows of this table right so posgressql of course is more advanced is is a great great database but if you're writing a small application with not a lot traffic and not a lot of like conflicts, you can uh uh very well live with SQLite and SQLite will take care of those collisions. So I I mean you're using fast API for example from uh Python and you can make it async and everything will work. So you don't have to be afraid of of collisions if if your traffic is not like very high. Okay. AI ready construction drawings. Yeah, this is a interesting video I bumped into. So the problem is that when you have architectural drawings, they are very very difficult for regular model to decipher. But you can do it in two steps. You can first use a process which uh takes the let's say PDF of those drawings and indexes it. It kind of decipher and and create different presentations. So you see workflow that performs a one-time index split sheets extract vector text map objects such as slabs foot and so on and once you create this index then you can talk to the model asking you questions and it can answer uh and you see the reported testing found 100% accuracy across 44 questions and uh so yeah this is great but you need so your model will not work out of the box. you need first to allow it to create this uh index. You need to pre-process the data. Okay. Uh next claude plus blender uh for character animations. So the these are the examples and how it works. So you have your claude uh on your computer. So let's say you're using claude desktop. Then you start local Blender MCP server, right? And then uh Blender has an add-on uh inside the running Blender and then you can work with it. Uh so it's well it's great because now you can create animations using text prompts. So this is the process. Install Blender. Download Adon py from Blender MCP. Install UV. Add the MCP server entry to desktop configuration. Open Blender. tell it to use Blender MCP and then you can work uh great uh V IO AI video production. So it's all in one uh video platform for doing uh different operations creating editing captioning dubbing and branding. Genai studio can turn a rough prompt or existing script into a scene based draft with narration visuals and so on. choose presets, animates, uh, builtin AI editing, includes transcript based cuts, audio cleanup, translations, dubbing, stalkers, and and so on. So, this is a great system. So, it's a vid.io uh website as you see the links here. Uh, next, Warp Rust based terminal for AI development. Well, we spoke about WARP multiple times. It's a very popular and very effective system uh to do AI uh coding, AI development. It's open source. It is a Rust based. It's basically a terminal based uh UI uh for agentic development. Work on Mac OS, Linux, Windows. Uh command blocks keep an entered command in its output together. Editor style terminal built in. Okay. Warp can host its own coding agent or coordinate uh things like cloth code, codex and gemini. WAP drive provides reusable workflows or it's in orchestration layer for running agents locally. Okay. Warp uh do you need Kubernetes? uh you know uh um I don't know I never used Kubernetes myself and I'm not uh uh planning to uh but uh it's interesting question because I see people use clusters of servers just for the heck of it when you really don't need it and the overall trend is to go from distributed systems uh to monolithic systems right Kubernetes definitely has its face. So here in uh blue, it fits high availability system spanning multiple nodes, zones, clusters. It's good fit for shared internal platforms with multiple teams, tenants uh and so on. It's available for hybrid on premises agent. It remains strong for large uh systems. So it still has its place, but think hard before you you decide to use it. And by the way, Kubernetes is um commonly called K8S. So this is Kubernetes and uh this is K then eight characters and then S. And that's why it's called K88S. Anyway, uh next uh five lessons from teaching more than 3,000 people to use AI in 12 weeks. Yeah, this is a very interesting YouTube video. Uh and this is kind of the steps of the process. Build a portable to agnostic second brain uh like a maybe obsidian type wiki with a deep business context, project records, customer profiles, pricing roles and so on. Standard operating procedures. Start with a pain, not a chatbot. Identify repetitive uh disliked work. Trace its course and TAI document to handle it. Use voice. Yeah, this is important. This is what Andre Karpatha recommended. Just talk to the system. Uh brainstorm with it. Explain the problems. Use recording and transcripts to capture expertise. Improve context. Turn meetings into proposals, follow-ups, and workflows. Invest in skills and capable plans. Don't wait until somebody will tell you to do it. Experiment, but do it safely. And expect initial productive struggle before automation pays off. This AI first mindset can shift work from execution to review, creativity, client work and higher value decisions. This is actually very very good video. So you see the the plan the context. Let's say it's your wiki identifying the pain talking to the system. You start the project uh you're talking to it and it can it use this information and then you put it in practice and then you provide the value. Oh, here CMAX uh every time now I talk about CMAX. It's a great system. It's only Mac OS, but this is for running multiple agents in the same terminal application. Uh so this is a great system. Okay, Hermas bot mode in Hermas desktop. Uh so what's happening? You can create multiple agents with different personalities. So it organizes persistent named AI bots with distinct roles. It's MIT license. Uh so it's uh free open source support local and provide the selected models. Okay. Hermes agents go portable. What it means that you can do export and import commands. Uh the package preserves persona memories learn skills schedules plug-in settings and the stop appearances. credentials, API keys are stripped away automatically, stripped off so recipients authenticate with their own accounts. Uh yeah, so this is a great way to share uh your uh stuff. Okay. Uh I love Math Burman. uh watch all his videos and uh this is a very useful video where he shares several open-source AI tools uh unsloth diagram design obsidian skills buzz we spoke about buzz before eagleite and uh modley local image to 3D mesh generation okay you can so these are the links you can review it on your own time um autonomous LLM optimization so custom LLM optimization ation. So let's say you have a specific customer and they have a model [snorts] and they have their own traffic. Uh how you can optimize the delivery for this particular customer for this particular situation and this is a publication. This is an article you see research inference optimization which explains uh how to do it starting from selection of servers and GPUs and how you distribute stuff and by this custom tuning of the hardware and software. You see they raise small model throughput from 165 to nearly 300 tokens per second. So this just shows practically uh that you can make things better. Now Claude obsidian a free MIT licensed claude code obsidian system turns local markdown notes into linked knowledge vault. So this is I think third or fourth uh piece of software which uh I'm talking about. I do the same thing when I create my own obsidian wiki. uh but I don't use any uh like third party skills. I just do it uh myself. Uh but uh here you you you can use this GitHub uh repo. So it uh already have pieces to ingest sources through inbox cloud organize cross links retain source. Okay. Evidence first QA designed to answer only the vault material and so on. So this may be a very good uh starting point if you want to create your own uh obsidian system. Uh uh AI updates. Okay. Cursor, Cordex, CL, Z, anti-gravity. So these are five very famous uh tools. Uh anti-gravity of course is from Google and uh they all good and they all have their uh strong features like Z for example is very fast. It's written in Rust and it's very lightweight uh but doesn't have many extensions as maybe others have a cursor very mature many models extensions browser tools code review multi- aent workflows Google anti-gravity uh many people love it manages isolated parallel workspaces and tightly integrates Germany chrome automation and so on very good for user interfaces openai codex and cloud code uh everybody's favorite dedicated model usage CLI cloud workflows work isolation MCP support uh so the reviewer's preference is cursor and codex anyway dots three node long horizon AI model and uh dot studio so this is coming from China it is Apache so it's open model 280 billion parameters mixture of expert uh half a million tok in context window works to text, images, video and audio and available on open router although only text and image no video and audio on open router and uh yeah so this is uh a great model uh corser origin so Corsa now has its own model which is very affordable very good and now they have their own uh kind of version of GitHub so it's very raw It's only when people use cursor. So it it's inside inside courser. But this is uh a git code hosting. So uh it's version control which is built into cursor world. It supports repositories, code browsing, pull requests, reviews and so on. Existing GitHub repositories can sync into origin. Origin targets agent scale development. uh launch timing coincided with major GitHub outage. There was a 8hour outage in GitHub. Uh so initially available only to paid cursor users. So cursor origin uh Andrew Ang 2hour course uh this is about leng graph uh explaining how to use it. lengraph. I was very disappointed two years ago when I tried to use lench chain and leng graph because uh of many layers of abstraction and because the documentation was always not in sync with actual code and it's very difficult to debug. But today when you're using agents uh agents can actually do this technical stuff for you and the system by itself is very good. There are millions and millions of downloads. It's a production grade. People use it everywhere. So what you have you have uh nodes, edges, conditional edges. Uh so you you present your workflow as a graph uh in Python, let's say. And uh this this is a good two-hour tutorial which walks you through uh how to work with lengraph. Uh next uh uh so here Andrew Ang's team uh read over 10,000 AI job ads and they identified what's in in demand. So building and deploying AI applications it's number one software engineering fundamentals still required using coding agents of course and shaping the build deploying the application. So these are 1 2 3 four main skills which are in highest demands. Uh indeed com reports software development posting rose nearly 15% after cloud codes launch. So 71% in senior roles. So mostly a as you we discussed before the junior roles outsourced to AI but senior roles not uh declined but actually increased. Okay, the demand for senior software engineers. Okay, these are layoffs in August. Not many layoffs. And this is me as usual. And thank you.