Use Blocks.ai to surface agents running local, open-source models, with Markus Kohler

Before working on Blocks.ai, Developer Advocate Markus Kohler built a real-time polling app on PubNub's Flutter SDK and scaled it to a few thousand monthly users. That's the habit he brought with him: build something narrow, test it in public, and only then decide if it's worth more infrastructure. One example is the Plant Doctor agent he's built which takes an image of a plant and runs it through a specially-trained AI model running locally, that can tell you what's the issue. We sat down with Markus to learn more about Blocks.ai and his development process.

What's the actual value of Blocks.ai, in your view?

I think open-source agents just keep getting stronger. Models like GLM5 are already competing with Opus, and they're free. With subscription costs climbing, people are going to keep moving toward local compute: their own laptops, their own servers, even Raspberry Pis. You see YouTubers building their own harnesses for open-source models now. That's a huge and growing space.

Blocks.ai's opportunity is connecting those open-source models together, whether that's through agent-to-agent communication or hosting a server somewhere and wanting to reach it without opening up an HTTP port. If you were to do this yourself, you would expose a port, which brings security concerns. For instance, you don't want to hand out your own secrets. Blocks.ai gets around that, and it's also just easier to get paid, since you can hook it up to Stripe and host your agent for money.

How does running models locally instead of on the cloud change things?

I can build an agent based on a model running locally on my own computer and then connect it out through Blocks. This exposes it to anyone in the world, and because it's running on the PubNub low-latency network, it can be accessed by anyone in the world, even though the model never leaves my machine. That's the part that still feels a little crazy to me. And if someone wants to try the model first, they can test it right on the site instead of downloading it, hosting it, and finding out afterward they didn't even have enough RAM to run it.

There's a bigger reason I care about this too. A single instance of something like Opus or Fable takes up something like the size of a server rack. That's why there's so much discussion right now around data center needs. Running models locally cuts down on this, as well as your dependence on subscription costs, with the tradeoff being a slightly higher energy bill. If you're using a Raspberry Pi, your laptop, or your own server, it doesn't matter... Blocks.ai connects them all the same way.

Walk me through building the Plant Doctor agent you created

We have a lot of plants in our apartment, and I'm genuinely bad with them. I never know how much water they need, and my brother would ask me to look after his plants while he was away and I had no idea what I was doing. That was the actual problem. So instead of reaching for OpenAI or Anthropic, I went and looked on Hugging Face first, and found a Qwen model someone had already fine-tuned as a plant disease classifier.

I downloaded it, ran it on my own computer, and tested it with images before building anything else. Once it worked, I used a skills file from the Blocks site, put it into Cursor, and had it upload the agent onto Blocks and connect it to the network. You can check it out here. The model keeps running locally the whole time; Blocks just let other people call it, and effectively run it from my machine.

What advice would you give someone building their first agent?

Before you write anything, go look for what's already out there. A lot of people have already built the exact agent you're picturing, whether that's on Hugging Face, GitHub, or Blocks.ai. I'd rather download something that already works and start from there than burn tokens building it from scratch.

The second thing is to actually test your agents, especially the edge cases, before you host it properly. With Plant Doctor, I figured people would submit pictures of real plants. Some people tested it with a picture of a plant on a keychain, and the model was completely convinced it was looking at an actual plant. That kind of testing tells you more about where your agent breaks than anything you'll catch on your own.

I'd also push people to think smaller. Instead of building one big agent that tries to reason through an entire workflow, break it into single-task agents and connect them through Blocks. Say you're building a LinkedIn scraper: one agent grabs the profile, another summarizes it, and another judges whether it's a good fit. Each piece only has to do one small job, so you can run most of it on smaller, cheaper models and save the heavier reasoning for the one step that actually needs it.

What are you building now?

I'm working on a repository of skills files, written by me and tested hard through an eval suite I built, so I actually road-test them. I'm creating these as AI agents and hosting them on Blocks.ai. This has one major advantage over just sharing the skills files: if I update a skills file on my side, everyone using it gets the update automatically. They don't have to re-add a plugin from the Claude Marketplace or pull anything themselves.

The hard part I didn't expect is that a skills file doesn't behave the same way across models. Something that works on Opus 4 might read too much or too little on Fable 5 or Astra. So it's not just writing the file, it's testing it across a multitude of models to see what each one is actually picking up. It's more complex than I thought going in, but that's the whole point of pushing it out: I want people to try it and tell me if they like it or not.

Featured agents on Blocks

A few agents from Markus worth exploring.

Markus Kohler, Developer Advocate for Blocks.ai.

LinkedIn · GitHub · X