Rick van Lieshout ~ Mastermindzh

Skillful, reusable AI workflows from files you own

Mon, 11 May 2026 00:00:00 GMT

A few weeks ago I started building a tool called Skillful.

It didn’t start as a product idea.

It started as friction.

The Problem: AI Workflows are everywhere and nowhere

If you use AI tools regularly, you probably recognise this:

prompts live in ChatGPT
some live in Cursor
some are buried in Obsidian
some are in random .md files
screenshots live somewhere else entirely and you copy-paste them into your prompt when required
helper scripts are in a repo you forgot about

Everything works… until you try to reuse it. Or worse, until you try to share it across tools.

That’s where things fall apart.

You end up copying the same workflow five times:

once for ChatGPT
once for Cursor
once for a markdown note
once for a repo
once for “just in case”

And now you have five slightly different versions.

None of them are “the source of truth”.

The Idea: Files as the Source Of Truth

Skillful is built around a very simple idea:

Your workflows should just be files.

Not a database. Not a SaaS backend. Not tied to a single tool.

Just files on disk.

Each “skill” or “agent” is a folder.

Inside that folder you can have:

prompts
markdown notes
screenshots
helper scripts
metadata

And that folder becomes the single source of truth.

From there, Skillful handles:

installing it into tools (Cursor, VS Code, etc.)
linking everything together
keeping things in sync

No copies. No drift.

Filesystem-First By Design

This is probably the most important architectural choice.

Skillful is filesystem-first.

That means:

your data stays on your machine
no accounts
no sync service
no telemetry
no lock-in

If Skillful disappears tomorrow, your files are still there.

You can open them with any editor. Version them with Git. Move them anywhere.

Skillful is just a layer on top.

Not a dependency.

How I Actually Use It

The easiest way to understand Skillful is through usage.

1. Create A Skill

I start with a folder:

skills/
  api-review/
    SKILL.md
    notes.md
    examples/
    scripts/

Inside SKILL.md lives the actual prompt I use. notes.md contains context, edge cases, things I learned.

Sometimes I add:

sample inputs
expected outputs
helper scripts

That folder becomes a reusable unit.

2. Link It To Tools

Skillful lets me “install” that skill into different tools.

Instead of copying the prompt, it links it.

So:

Codex sees it
Claude sees it
other tools see it
updates propagate automatically

Change the file once, everything updates.

That’s the key difference.

3. Reuse Everywhere

Now the same workflow can be used:

inside editors
inside AI tools
inside scripts
inside repos

Without duplication.

Without drift.

One Source, Every Tool

One thing I didn’t expect initially is how valuable this becomes over time.

Once you have a growing library of skills:

you stop rewriting prompts
you start refining them
you start composing them
you start trusting them

And because everything is file-based, it’s trivial to:

version control them
share them
review them
evolve them

It starts feeling less like prompts and more like actual software.

Why Not Just Use Notion / ChatGPT / X?

Because those tools are not designed for reuse across environments.

They are designed for:

interaction
storage
sometimes collaboration

But not for:

cross-tool installation
filesystem ownership
reproducibility
composability

Skillful is not trying to replace those tools.

It sits next to them.

It makes the workflows portable.

Break-Proof Installs

One small but important feature is how installs behave.

Instead of copying files into tools, Skillful links them.

That means:

no duplication
no stale versions
no mystery bugs from “which version is this?”

And if something breaks?

You can repair it in a click.

Or just delete and re-link.

Because the source is always your folder.

Your Library, Your Disk

This is probably the most important part philosophically.

Skillful does not own your data.

You do.

There is:

no cloud dependency
no vendor lock-in
no hidden sync
no background services

Everything stays local.

Which also means:

it works offline
it’s fast
it’s predictable

Final Thoughts

Skillful isn’t trying to be a massive platform.

It’s trying to solve one very specific problem:

How do I make AI workflows reusable without losing control over them?

For me, the answer ended up being surprisingly simple:

Files.

Plain files. On disk. As the source of truth.

Everything else builds on top of that. If that resonates with how you work, you can check it out here: Skillful

And if nothing else, maybe it gives you a different way to think about managing your own workflows.

I wanted a lighter runtime like Electrobun, shipping reality chose Electron

Sun, 10 May 2026 00:00:00 GMT

About a month ago I started building a new application called Skillful).
Originally, the idea was quite simple: I wanted an excuse to try Electrobun.

For those unfamiliar with it, Electrobun is essentially an Electron-like desktop runtime built around Bun instead of Chromium + Node.js. The biggest selling point is obvious immediately: no embedded Chromium browser. Smaller runtime, potentially lower memory usage, less “Electron bloat”.

And honestly? That attracted me.

People love complaining about Electron applications shipping half a browser with every app. Personally, I’ve always thought that criticism was slightly overblown. RAM exists to be used, and modern desktop applications are expected to behave like modern applications. Still, the idea of a lightweight filesystem-first desktop app built on Bun sounded genuinely interesting.

So I gave it a shot.

At first, it actually went pretty well.

Then I tried to ship it.

The Reality Of Desktop Applications

One thing became painfully clear very quickly:

Building a desktop application is not the hard part anymore. Shipping one is.

Packaging, updating, shortcuts, desktop integration, installers, Linux distributions, release channels, runtime quirks, auto-updates, platform conventions… that is the real product surface of a desktop application.

And this is where Electrobun started falling apart.
Not because it was fundamentally bad. In fact, the runtime itself was perfectly fine most of the time.

The problem was that I kept ending up in runtime land instead of product land.

Issues looked like product bugs but turned out to be packaging problems. Or updater problems. Or launcher problems. Or Linux runtime problems.

Every single one costs time, every single one is annoying.

Why I Chose Electrobun

Before I explain why I moved away from it, it’s important to explain why I chose it in the first place.

Skillful is intentionally filesystem-first.

The application manages local AI skills, local models, local assets and local workflows. The disk is effectively the source of truth. I wanted an actual desktop application that happened to have a modern UI, normally I’d use Electron but I wanted to try Electrobun. (hell, the whole idea of Skillful was born because I wanted to use Electrobun)

Electrobun felt ideologically aligned with that.

Smaller runtime.
Fast startup.
Native feeling.
Bun tooling.
No Chromium baggage.

And architecturally, it fit extremely well. And it also delivered on speed, the filesystem was fast without me applying any efficiency standards. It truly felt well.

One of the best decisions I made early on was separating the runtime shell from the actual application logic.
The project structure looked roughly like this:

src/
  main/           -> business logic
  shared/         -> shared RPC + contracts
  mainview/       -> renderer/frontend
  desktop/        -> runtime abstraction layer

The important part here is that src/main/ never cared about the runtime.

Filesystem scanning.
Libraries.
Archives.
Settings.
Collections.
AI skill metadata.

All of that was runtime-agnostic. The runtime only needed to satisfy a few interfaces:

open file dialogs
reveal paths in the file manager
perform updates
expose IPC
manage windows

That architectural seam ended up saving the project later. At the time though, it just felt like good engineering hygiene.

The Packaging Disaster Begins

The first warning signs appeared once I started packaging builds. Every platform had sharp edges.

Windows especially became a rabbit hole of custom glue code and weird packaging hacks.

I ended up maintaining:

custom launcher scripts
bundled bin/bun
bundled runtime files
zstd-compressed payloads
manual patch generation
update archive handling
custom extraction logic

And the frustrating part is:

none of this had anything to do with Skillful itself.

I wasn’t building features anymore. I wasn’t building infrastructure around the runtime.

The Linux packaging story also became especially painful because there wasn’t really a conventional ecosystem around Electrobun releases yet, worse, it doesn’t align with other Linux packaging guidelines.

Electron, meanwhile, has had years of people solving these exact problems already. That difference matters far more than people think.

Electron Won By Being Boring

Eventually I switched the project back to Electron.

Not because Electron was elegant. Not because Electron was lightweight. But because Electron made the difficult parts boring again.

And boring infrastructure is one of the best things you can have.

Using electron-builder, a single configuration now produces:

macOS .dmg and .zip
Windows NSIS installers
Linux .deb, .rpm, .AppImage, .pacman, .snap, .flatpak

That is an absurd amount of packaging complexity solved by one ecosystem. Previously the Linux story was basically:

“here’s a tarball, good luck.”

Now releases are conventional.
Predictable.
Expected.

That matters.

Desktop Integration Should Not Be A Side Quest

Another thing Electron completely solved was native desktop integration. Things I spent days wrestling with before became one-liners:

dialog.showOpenDialog()
shell.showItemInFolder(path)
shell.openExternal(url)

Icons?
Handled.

Start menu entries?
Handled.

Desktop files?
Handled.

Uninstallers?
Handled. (and actually used.. Electrobun produced an Uninstall.reg but never applied it)

I cannot overstate how valuable it is when platform integration stops being your problem.

The Funny IPC Bug

Ironically, one of the most interesting bugs happened after migrating. Electron prepends this wonderful little string to IPC errors:

Error invoking remote method 'rpc:call':

Which sounds harmless. Except the domain-specific error transport depended on a sentinel existing at the start of error.message. So suddenly the custom AppError rehydration stopped working.

The fix itself was hilariously small:

startsWith(...)

became:

indexOf(...) >= 0

One-line fix.

But the important part was the lesson:

runtime-specific bugs absolutely exist, and the only reason I caught this one was because I ran tests against the actual packaged runtime.

Not just unit tests.

Real packaged application tests.

Without that, I probably would have shipped it without the fix first.

The Runtime Seam Saved me

The best thing to come out of the Electrobun experiment was honestly the runtime abstraction layer.

Switching runtimes ended up taking about a single evening.

Not because Electron migration is inherently easy, but because the architecture isolated the runtime shell from the actual application.

The domain layer stayed untouched.

The renderer stayed mostly untouched.

The filesystem model stayed untouched.

The only things rewritten were:

runtime adapters
packaging
IPC wiring

That is exactly what you want from architecture. The runtime became an implementation detail. Which, honestly, it should have been from day one.

What I Learned

Looking back, I think the biggest lesson here is this:

Runtime minimalism is a false economy if you end up rebuilding the ecosystem yourself.

Saving some memory usage on paper is meaningless if your release pipeline needs babysitting every week. Electron absolutely uses more memory. Nobody serious denies that. But the application now:

ships reliably
updates reliably
packages reliably
integrates with the OS properly
behaves conventionally
is dramatically easier to reason about

That trade-off was worth it, instantly. And funnily enough, the filesystem-first architecture survived the migration almost untouched.

Because filesystem-first was never really about Electrobun.

It was an architectural decision. Not a runtime one.

Final Thoughts

If I had to do it all again?

I’d probably start on Electron immediately. Not because Electron is technologically superior. But because desktop application development is overwhelmingly operational work once the product itself exists.

The runtime you choose should make packaging, updates and desktop integration boring.

Electron does.

And after spending weeks deep inside launcher scripts, patch files and packaging edge cases… boring started sounding really, really attractive.

Would I Use Electrobun Again?

Honestly?

Yes, probably.

Despite everything in this post, I actually still like the idea behind Electrobun a lot.

The runtime itself was not the problem. In many ways it was refreshingly straightforward. Startup times felt great, Bun is genuinely pleasant to work with, and the smaller runtime story still appeals to me philosophically.

But desktop applications are judged on the parts around the runtime.

Users do not care what JavaScript engine powers your application. They care whether:

installing works
updating works
file associations work
uninstalling works
icons work
shortcuts work
the application feels native

Electron has years of ecosystem maturity solving exactly those things. Electrobun is simply not there yet. I would absolutely consider using it again once a few things mature further.

Native Packaging Needs To Become The Default

The biggest one for me is installation strategy. Desktop applications should install using conventional OS-native mechanisms.

Not:

“extract this tarball somewhere and run the launcher.”

That may be technically acceptable for developers, but it is not acceptable for mainstream desktop software. The moment your installer story becomes custom, you inherit years of operational complexity yourself.
Electron ecosystems solved this years ago.

Desktop Integration Needs To Be Assumed

Basic OS integration should not require detective work.

Things like:

application icons
Start Menu entries
uninstall registration
desktop files
external URL opening
native dialogs

should simply work.

Not “work with a helper script”. Not “work if you manually patch the generated output”. Not “work after reading three GitHub issues”.

Just work.

One of the nicest feelings during the Electron migration was deleting workaround code instead of writing more of it. That alone said a lot.

The Chromium Embedded Suggestion Scared Me

One thing that genuinely surprised me was seeing the documentation suggest bundling Chromium Embedded Framework (CEF) for Linux packaging scenarios.

At that point I had a bit of an existential moment. Because if the solution to packaging problems becomes:

“ship Chromium anyway”

then we’ve effectively lost the entire original value proposition. The whole reason I explored Electrobun was to avoid shipping massive browser runtimes in the first place.

Once we start reintroducing pieces of Chromium manually, things become very difficult to justify operationally.

The Ecosystem Needs Time

And honestly, this is probably the fairest conclusion.

Electron is old. Very old.

Which means:

millions of edge cases already happened
somebody already solved most packaging problems
updater pipelines are battle-tested
CI examples exist for everything
installers are predictable
desktop conventions are well understood

Electrobun simply has not had time to accumulate that maturity yet. That is normal.
I think the project has potential. I genuinely do.

But for Skillful, where reliable releases mattered more than runtime ideology, maturity beat elegance. And right now, Electron is still overwhelmingly the more mature choice.

The Reality of Distributed Systems: Everything Gets Harder

Thu, 26 Feb 2026 00:00:00 GMT

In the previous articles we explored how systems evolve, how they communicate, and where to draw boundaries. All of those decisions eventually lead to one moment:

The moment where your system is no longer a single unit. The moment where it becomes distributed.

On paper, that often looks like progress. More scalable, more flexible, more future-proof. That might well be true, but in practice, it is also the moment where everything gets harder.

The moment things change

Moving to a distributed system does not feel like a big shift at first. You extract a service. You move some logic out. You introduce an event or an API call. Everything still works. The system behaves roughly the same.

Until it doesn’t.

The difference is subtle, but fundamental. You are no longer working within a single process. You are working across network boundaries. That means you are no longer dealing with function calls. You are dealing with unreliable communication.

And that changes everything.

Debugging is no longer local

In a monolith, debugging is straightforward. Something goes wrong, you follow the stack trace, inspect the state, and you usually find your answer.

In a distributed system, there is no single place to look. A request might start in one service, pass through two others, trigger an asynchronous process, and eventually fail somewhere completely unexpected. By the time you notice, the original context is gone.

Now you are searching across logs, correlating timestamps, and trying to reconstruct what actually happened. This is where tools like tracing and telemetry become essential. Not because they are nice to have, but because without them, you are effectively blind. Even with good tooling, it takes time to build the mental model needed to debug these systems.

I have seen this firsthand in every place I worked with a distributed system.

When new engineers join, especially those coming from more traditional or monolithic environments, this is one of the hardest things to teach. It is not just about reading logs, it is about understanding flows that are no longer linear. We have had to invest in internal sessions, tooling like SigNoz, and shared practices just to get people comfortable with distributed debugging.

And even then, it takes time.

Failure is everywhere

In a monolith, failure is usually binary. Something works, or it throws an error.

In a distributed system, failure is everywhere, and often partial. A service might respond slowly. Another might time out. A third might process a message twice. A fourth might never process it at all. None of these are exceptional situations. They are normal.

This forces you to think differently..

You need retries, but retries can cause duplicates.
You need idempotency, but that adds complexity.
You need timeouts, but choosing them is not trivial.

You are no longer designing for success. You are designing for failure. And if you ignore that, your system will behave unpredictably under real-world conditions.

Time becomes a problem

In a single system, time is mostly implicit. Operations happen in sequence. State changes are immediate. In a distributed system, time becomes one of the hardest problems to reason about. Messages can arrive out of order. Events can be delayed. Systems can temporarily disagree on what the current state is.

This is where eventual consistency stops being a nice concept and becomes a real constraint. You might process an order, publish an event, and have another service react to it seconds later. Or minutes. Or after a retry. If your system assumes everything is always in sync, it will break.

Designing for this means accepting that different parts of your system will see different realities at different times. That is not a bug. That is the system working as designed.

Observability is not optional

In a distributed system, logging alone is not enough. You need to be able to answer questions like:

Where did this request go?
Which services were involved?
Where did it slow down?
Where did it fail?

This requires more than logs. It requires structured logging, tracing, correlation IDs, and metrics that actually tell a story. Without that, you are guessing. With it, you are still working, but at least you are working with context.

The difference between those two states is enormous.

The team impact is real

This is where things often get underestimated.

Distributed systems are not just harder technically. They are harder for teams. Not only does the context of the application become larger, and thus harder to grasp. But onboarding new members (or colleagues!) becomes more difficult. New engineers need to understand not just code, but interactions between multiple services, asynchronous flows, and failure scenarios.

We have seen this very clearly.

As our system evolved, we had to adjust how we hire and onboard at Frontliners. It was no longer enough to look for someone who knew the stack. We needed people who could adapt, reason about systems, and build a mental model of something that is not immediately visible.

We also had to upskill existing teams. For example, engineers coming from environments like Clarion or ASP.NET had to learn not just a new language like TypeScript, but a completely different way of thinking about systems. Distributed debugging, eventual consistency, and asynchronous flows are not things you pick up overnight.

This takes deliberate effort.

Training, internal sessions, shared knowledge, and a culture where asking questions is encouraged all become essential.

The trap

One of the biggest traps in architecture is underestimating this shift. It is easy to say “we will figure it out later” or “we will add retries” or “we will just log everything”.

These are not solutions. They are placeholders.

Adding retries without thinking about idempotency creates duplicate processing.
Logging everything without structure creates noise, not insight.
Ignoring failure modes works, until it doesn’t.

Another common trap is distributing too early. If your system does not yet need the benefits of distribution, you are taking on complexity without getting much in return. Simplicity is still a valid choice.

Just like with the modular monolith, starting simple is often the right move. If you reach a point where that simplicity becomes a bottleneck, that is a good problem to have.

But if you start complex, you are committing to a problem space that requires experience, discipline, and tooling to handle well. I’m not saying you shouldn’t… your situation might require it. But best be prepared!

Visualising the difference

A grounded takeaway

Distributed systems are powerful. Very powerful.
They allow you to scale, to decouple, and to build systems that can evolve independently.

But they demand more from you.

More discipline.
More awareness.
More investment in tooling and people.

They are not something you casually adopt. If you are not ready to deal with failure, time, and complexity as first-class concerns, you are better off staying simple for a little longer. And that is not a step back. That is a conscious decision.

Wrapping up

Moving to a distributed system is not just a technical change. It is a shift in how you think about software.

You stop assuming things will work, and start designing for when they don’t.
You stop relying on immediate consistency, and start working with eventual outcomes.
You stop debugging locally, and start reasoning about flows.

That shift takes time. It takes experience. And it takes a willingness to accept that things are no longer simple.

Boundaries: Where to Draw the Line

Sun, 15 Feb 2026 00:00:00 GMT

In the previous articles we explored how systems evolve structurally, and how they communicate. Both of those topics depend on one underlying concept: boundaries.
Where you draw them determines how your system behaves.

Get them right, and everything feels simpler. Get them wrong, and no amount of tooling, patterns, or infrastructure will save you.

Most problems are boundary problems

A lot of architectural discussions focus on technology choices. Frameworks, messaging systems, deployment strategies. After all, that is also really fun to talk about. In practice, however, most real problems have very little to do with those things.

They come from boundaries that were drawn in the wrong place.

Services that depend on each other for basic logic. Data that is shared across parts of the system that were meant to be independent. Teams stepping on each other’s toes because ownership is unclear. Or the most frustrating one: systems that were split up, but nothing actually became simpler.

That is almost always a boundary problem.

What is a boundary, really?

A boundary is not a folder, a repository, or a service.

It is a decision about ownership.

Who owns this logic?
Who owns this data?
Who decides how it evolves?

You can draw boundaries inside a modular monolith, or across multiple services. The shape does not matter as much as the clarity. A good boundary reduces the need for communication. A bad boundary creates it.

Start from the core

In many systems, there is a central process that everything revolves around. At my current employer’s world, that is planning.

Customers are planning how to move loads from one place to another. They are optimising routes, combining shipments, and trying to make that process as efficient as possible.

That is the core of the business.

When boundaries are not defined carefully, everything starts to leak into that core. Suddenly, planning is responsible for things it should not care about. Notifications, integrations, data transformations, and edge cases all start piling up. At that point, it becomes harder to change, harder to understand, and harder to scale.

The goal of boundaries is not to split everything evenly. It is to protect the core.

Data ownership is the real boundary

One of the clearest ways to define a boundary is through data ownership.

If two parts of your system share the same data freely, they are not truly separate. They are coupled, whether you acknowledge it or not.

That is why a strong default is simple:

Each boundary owns its data.

That does not mean you can never be pragmatic. There are situations where sharing data is the least bad option. But those should be conscious decisions, not the default.

If you start from shared data and try to separate later, you will feel the pain immediately. If you start with ownership, you at least have the option to relax it when needed.

Teams define boundaries, whether you like it or not

Architecture does not exist in isolation.

The way your teams are structured will shape your system. This is often referred to as Conway’s Law, but you do not need the name to see it in practice… If multiple teams need to work on the same part of the system every day, your boundaries are probably wrong.

If one team owns a domain end-to-end, including its data and behaviour, things tend to move faster and cleaner.

This also ties into maturity.

A highly experienced team can handle more complex boundaries, more decoupling, and more asynchronous communication. A less experienced team will struggle if the system requires too much coordination or implicit knowledge.

Good architecture meets teams where they are, not where you wish they were.

Where things go wrong

There are a few patterns that show up again and again. One of the most common is “everything is a service”. Every feature becomes its own service, often without clear ownership or responsibility. The system looks nicely split on paper, but in reality everything depends on everything else. You end up with a distributed system that behaves like a monolith, but with all the downsides of distribution.

Another common issue is shared databases across supposed boundaries.
Two services, two repositories, maybe even two teams, all reading and writing to the same tables. At that point, the separation is an illusion. Changes ripple through the system in unpredictable ways, and coordination becomes unavoidable.

Then there are boundaries defined by technology instead of domain.
A service for the API, a service for the database, a service for background jobs. These are technical slices, not business boundaries. They tend to create more communication, not less.

And finally, the most subtle one: boundaries that looked right at the time, but were never revisited.
Systems evolve. Teams evolve. What made sense a year ago might not make sense today. If boundaries stay fixed whilst everything else changes, they slowly become a source of friction.

Drawing better boundaries

There is no formula that guarantees the right answer, but there are good starting points:

Look at your core processes. Where does the business logic naturally group together?
Look at your data. Who should own it, and who should not?
Look at your teams. Where can you give ownership without constant coordination?
And most importantly, look at where your system hurts today. Boundaries should reduce that pain, not introduce new kinds of it.

It is an ongoing process

Just like everything else in architecture, boundaries are not a one-time decision. They shift over time.

As your system grows, you will discover that some boundaries were too broad, others too narrow. Some will need to be split, others merged. That is normal.

The goal is not to get it perfect from the start. The goal is to make decisions that can evolve.

Wrapping up

I repeat: Most architectural problems are boundary problems. When boundaries are clear, communication becomes intentional. Systems become easier to understand, easier to change, and easier to scale.

When boundaries are unclear, everything becomes harder. Not because the technology is wrong, but because the structure does not match the reality of the system.

Start with ownership. Protect your core. Be pragmatic when needed, but deliberate in your decisions.

And most importantly, revisit your boundaries over time. Because the system you are building today is not the system you will have a year from now.

In the next article, we will look at what it really means to run a distributed system, and why things that used to be simple suddenly become much harder.

Synchronous vs Asynchronous Communication: When to Call and When to Publish

Tue, 06 Jan 2026 00:00:00 GMT

In the previous article we explored the difference between data events and domain events, and how mixing them creates confusion. This time we take a step back and look at a broader question: how should services communicate in the first place?

Because once your system becomes distributed, communication is no longer an implementation detail. It is part of your architecture.

The false debate

Much like the monolith versus microservices discussion, synchronous versus asynchronous communication often turns into a debate.

You will hear things like:

“We should not call other services”
“Everything should be event-driven”
“Just make an API call, it is simpler”

None of these are universally correct.

They are simplifications of a much more nuanced problem.

Synchronous and asynchronous communication are not competing approaches. They solve different problems. Treating them as interchangeable, or worse, treating one as superior, is where things start to break down.

What synchronous communication really is

Synchronous communication is straightforward. One service calls another and waits for a response. A request goes out, a response comes back. This creates a direct dependency. Service A depends on Service B being available and responding in time.

That sounds like a downside, but it also provides clarity. Failures are immediate. Latency is visible. The flow of data is easy to follow. This makes synchronous communication a good fit when you need certainty.

For example:

Validating input before processing
Fetching data needed to complete a request
Checking permissions or business rules
Anything that requires an immediate answer

If you need to know something before you can continue, a synchronous call is usually the right choice.

What asynchronous communication really is

Asynchronous communication works differently. Instead of asking for something, you publish that something happened. You do not wait for a response. You move on.
Other services can react to that event in their own time, independently of the original request.

This removes the direct dependency between services, but introduces a different kind of complexity. Failures are no longer immediate. They can be delayed, retried, or even silently ignored if you are not careful.

That does not make asynchronous communication worse. It just makes it different. It shines in scenarios where you are reacting to something that already happened.

For example:

Sending emails or notifications
Updating search indexes
Triggering downstream processes
Integrating with external systems

In these cases, you do not need an immediate answer. You just need to make sure the reaction eventually happens.

A simple example

Consider placing an order.

The initial request might look like this:

Validate the order
Check inventory
Store the order

These steps are typically synchronous. You need to know whether they succeed before you can respond to the user.

Once the order is placed, a different set of actions follows:

Send a confirmation email
Update analytics
Notify other systems
Trigger fulfillment

These are not part of the immediate request. They are reactions to something that already happened.

This is where asynchronous communication fits naturally.

Trying to force everything into one model quickly becomes awkward.

If you make everything synchronous, your request chain grows longer and more fragile. If you make everything asynchronous, you lose control over what is actually completed when the request finishes.

Where things go wrong

Most issues do not come from choosing one approach over the other. They come from using the wrong one for the problem at hand. One common mistake is using asynchronous communication for things that require an immediate answer.

Publishing an event and hoping another service processes it in time is not a replacement for a direct call. It introduces uncertainty where you actually need guarantees. On the other side, teams often overuse synchronous calls across service boundaries.

A calls B, B calls C, C calls D. Everything works, until one service slows down or fails. Now your entire request chain is affected. This creates cascading failures that are difficult to debug and even harder to recover from.

There is also a more subtle problem…

Making something asynchronous does not make it simpler. It moves the complexity elsewhere. Retries, ordering, idempotency, and observability all become your responsibility. If you do not design for that, failures become invisible and behaviour becomes unpredictable.

The real trade-offs

Synchronous communication is easier to reason about. You see the full flow, you get immediate feedback, and failures are explicit. But it comes with tighter coupling and less flexibility. Services depend on each other being available at the same time.

Asynchronous communication decouples services and allows systems to scale more naturally. It enables independent processing and reduces direct dependencies. But it makes the system harder to understand. Debugging becomes more complex, and failures are no longer obvious.

Neither is better. They are trade-offs.

How they work together

In practice, most well-designed systems use both.

Synchronous communication is often used within a bounded context, where clarity and immediate feedback matter.
Asynchronous communication is used between contexts, where independence and scalability are more important.

A common pattern looks like this:

A request comes in
Services communicate synchronously to complete the core action
Once done, an event is published
Other services react asynchronously

This keeps the core flow predictable, whilst allowing the system to evolve and scale around it.

A simple rule

If there is one guideline that holds up in most cases, it is this:

If you need an answer now, make a call.
If something already happened, publish an event.

It is not perfect, but it is a good starting point.

Wrapping up

Just like with architectural styles, communication patterns are not about choosing the “right” one upfront. They are about understanding the problem you are solving.

Synchronous communication gives you certainty and simplicity. Asynchronous communication gives you flexibility and scalability. Good architecture is not about picking one. It is about knowing when to use each, and how they fit together.

Teams that treat this as a strict rule tend to struggle. Teams that understand the trade-offs tend to build systems that are easier to evolve. And just like everything else in architecture, this is not a one-time decision. It changes as your system grows.

In the next article, we will look at one of the hardest problems in software architecture: where to draw boundaries, and why getting that wrong hurts more than almost anything else.

Data Events vs Domain Events: Stop Mixing Them

Tue, 23 Dec 2025 00:00:00 GMT

In the previous article we explored how systems evolve structurally, from modular monoliths to distributed systems and eventually microservices. This time we move one level deeper, into how those systems communicate.

Once your system becomes distributed, communication is everything. This is where things often start to break down.

Where it usually goes wrong

Most teams do not set out to build a confusing event-driven system. They start simple. A change happens in a service, an event is published, and another service reacts.

Something like:

CustomerUpdated
OrderChanged
EntityUpdated

It feels clean at first. Generic, reusable, flexible.

Until it isn’t.

The moment another service consumes that event, it has to answer a question: what actually happened?

Was the address updated, the status changed, or something recalculated?

The consumer now has to figure that out. It pulls additional data, compares state, and often keeps local copies just to understand what changed over time.

At that point, meaning is no longer explicit. It has to be reconstructed.

That is where the pain begins.

The real issue is not events, it is ambiguity

Teams often blame event-driven architecture itself. Kafka becomes the problem. Asynchronous communication becomes the problem.

But it is neither.

The real issue is that the system is publishing data events, whilst consumers are expecting domain intent. That mismatch introduces complexity that spreads across the system.

Data events vs domain events

There is a simple but important distinction.

Data events describe what changed. Domain events describe what happened.

A data event looks like:

CustomerUpdated
OrderRowChanged
EntityUpdated

A domain event looks like:

CustomerAddressChanged
OrderPlaced
RouteRecalculated
InvoiceFinalised

One forces you to interpret. The other tells you exactly what to do.

That is the difference between extracting meaning from data and being given meaning directly.

Data events are not wrong

Data events are not a mistake. They are useful, and often necessary.

There are many scenarios where intent does not matter, only state. Filling dropdowns, synchronising reference data, updating read models, feeding search indexes, or keeping caches in sync.

In those cases, a simple “CustomerUpdated” event is perfectly fine. You do not care why something changed, only that you have the latest version.

This is where data events shine.

Where things go off the rails

Problems start when data events are used to drive behaviour.

When a service reacts to something like “EntityUpdated”, it needs context. Because that context is not in the event, it has to rebuild it. This leads to consumers fetching additional data, keeping their own state, and implementing logic that does not belong to them.

At that point, your system is no longer loosely coupled. It is tightly coupled through interpretation. You have not removed dependencies, you have just hidden them.

This is where “event-driven” quietly turns into async CRUD.

Domain events change the conversation

Domain events remove that ambiguity. They describe something that has already happened, in business terms.

OrderPlaced
RouteCalculated
InvoiceApproved

The consumer no longer needs to guess. It does not need to diff or fetch additional state just to understand intent. It simply reacts.

That is what event-driven systems are supposed to feel like.

Events should describe what happened, not what changed.

Kafka does not fix this for you

Kafka does not care whether your events are good or bad.

You can have clean topics, proper partitioning, and great throughput, and still end up with a system that is hard to reason about. The problem is not how events are transported, it is what they represent.

A topic full of “EntityUpdated” events is still a problem, no matter how well it scales.

It is not either or

In a well-designed system, you often have both.

You might publish a domain event like OrderPlaced, followed by a data event like OrderReadModelUpdated. Each serves a different purpose. One drives behaviour, the other synchronises state.

That is completely fine.

The key is that they are not the same thing, and should never be treated as such.

A simple rule

If there is one thing to take away, it is this:

Use data events to sync state. Use domain events to drive behaviour.

Why this matters more than it seems

As systems grow, these small decisions compound. A few ambiguous events turn into dozens, and a few consumers with extra logic turn into entire services built on assumptions.

Before long, your architecture becomes harder to change than the monolith you started with. Event-driven architecture is supposed to give you flexibility, but without clear intent it does the opposite.

Wrapping up

Good architecture is not just about structure. It is about clarity. Clarity in boundaries, ownership, and communication.

Events are one of the most powerful tools we have in distributed systems, but only if they actually communicate something meaningful.

As the year comes to an end, it is a good moment to reflect on the systems we build and the decisions we carry forward.

Take a step back and look at the events in your system. Ask yourself whether they describe what truly happened, or just what changed.

If there is room to improve, that is a good thing. It means your system is still evolving.

Wishing you a calm Christmas, and a new year filled with better decisions, simpler systems, and fewer EntityUpdated events.

Monoliths to Microservices: Finding the Right Moment

Thu, 06 Nov 2025 00:00:00 GMT

From simplicity to scale, and everything in between

In the previous article we explored what software architecture really is: a set of decisions that shape how a system evolves over time. This time we will look at how those decisions take shape in the structure of your system.

Most discussions around architecture quickly turn into debates. Monolith versus microservices. Centralised versus distributed. Right versus wrong. In practice, it is rarely that simple.

Systems evolve. Teams evolve. Context evolves. The shape of your architecture should evolve with it.

Start simple: the modular monolith

Almost every system benefits from starting as a modular monolith. One deployable, one runtime, but with clear internal boundaries.

That distinction matters. A monolith without structure becomes a problem. A modular monolith, with well-defined modules and ownership, gives you clarity without introducing unnecessary complexity.

You move fast and deploy easily, without the operational overhead of distributed systems. More importantly, you learn. You learn where your real boundaries are, which parts of your system change frequently, and how your team collaborates.

That learning is what enables the next step.

Of course, this only works if you take the modular part seriously. If everything can call everything, you have not built a modular monolith. You have just built a smaller version of your future problems. Clear boundaries, explicit interfaces, and discipline are essential.

This also applies to data. If every module shares the same database tables and freely reads from each other, your boundaries are not real. Separating database context per module, even within the same runtime, forces ownership and prevents accidental coupling.

It might feel like overhead early on, but it pays off later. When parts of your system need to move independently, those boundaries become the foundation for doing so.

A good problem: when your system starts to feel the pressure

At some point, a well-structured monolith will start to show strain. Not because it is poorly built, but because the system is growing.

You might see long-running tasks slowing down request handling, ingestion workloads increasing, or scheduled jobs competing with user traffic. Teams may start stepping on each other’s toes.

This is not a failure. It is a good problem to have.

It means your product is working.

The mistake many teams make at this point is jumping straight to microservices. There is a more natural step in between, which we’ll explore below.

Extracting services: evolving beyond the monolith

Instead of breaking everything apart, you start by extracting the parts that need it.

Think of sending emails in batches, running scheduled jobs, handling heavy ingestion pipelines, or offloading compute-heavy processes. These are often isolated concerns with clear inputs and outputs, making them good candidates to run independently.

Because your modular monolith already has boundaries, these parts can be moved out with relatively little friction. If a module owns its logic and data, extraction becomes straightforward. If it does not, hidden coupling quickly becomes visible.

At this stage, your system becomes partially distributed, but intentionally so. The monolith still handles the core domain, whilst specific responsibilities are moved into separate services that can scale and evolve independently.

This is not an accident. It is a system responding to real pressure.

It is also where anti-patterns can appear if you are not careful. Services without clear ownership, fake autonomy, or excessive communication are signs that boundaries are not yet well defined.

Microservices: autonomy comes at a cost

Microservices promise independence. Independent deployments, scaling, and ownership.

When done well, they deliver exactly that. But they are not free.

You introduce network boundaries, eventual consistency, and operational complexity that simply does not exist in a monolith. Observability becomes harder, debugging becomes harder, and understanding the system as a whole becomes harder.

Coordination does not disappear. It just changes form.

Where you once had compile-time dependencies, you now have runtime dependencies. Where you once had local calls, you now deal with contracts, versioning, and failure scenarios.

This is why starting with microservices is rarely a good idea. Unless you are building for massive scale from day one, you are taking on complexity you do not yet need.

It is far better to grow into microservices. Let the system show you where separation makes sense and move boundaries only when the benefit clearly outweighs the cost.

This shift introduces a different kind of complexity that teams often underestimate. Once everything is distributed, understanding what is happening becomes harder by default. Requests cross service boundaries, data flows asynchronously, and failures are no longer local.

That is why, even now, when new engineers join, one of the first things we teach them is how to debug a distributed system. Once you cross that boundary, everything changes.

It is not a switch, it is a spectrum

One of the biggest misconceptions is that systems are either monoliths or microservices.

In reality, most systems live somewhere in between. You might have a modular monolith at the core, a few extracted services for heavy workloads, and some fully independent services where autonomy is critical.

That is not a failure of design. It is a reflection of reality.

Different problems require different solutions. Forcing everything into a single model often creates more problems than it solves.

Choosing the right shape

So how do you decide?

You look at your context. How mature your team is, how well you understand the domain, where your system is under pressure, and what kind of change you expect.

Then you make a trade-off.

Start simple. Build a modular monolith with clear boundaries. Pay attention to where pressure builds and act just before it turns into pain.

My reflection on how we dealt with it at Inforit / Frontliners

Looking back, we did not follow this path cleanly.

We jumped into microservices early. There were ambitious plans that made it feel like the right thing to do, and with a small but strong team we managed to deliver multiple products in that setup.

But it came with a cost. The complexity was high, the learning curve was steep, and many of the problems we were solving were not yet problems we actually had.

We only really started addressing this properly at scale with the newer version of our system. That is where the focus shifted from building services to building something sustainable.

Wrapping up

Architecture is not about choosing the right structure upfront. It is about choosing the right structure for now, whilst preparing for what comes next.

Start simple. Let the system grow. Evolve your architecture just ahead of your problems, not far ahead of your understanding.

The moment where your modular monolith starts to struggle is not something to fear. It is a signal that you are building something meaningful.

And if you have prepared your boundaries well, that next step becomes a lot more manageable.

In the next article, we will look at what happens once your system becomes truly distributed, and how communication patterns shape everything that follows. We will dive into data events versus domain events, and why that distinction matters more than it first appears.

What is Software Architecture (Really)?

Tue, 14 Oct 2025 00:00:00 GMT

A new category on rickvanlieshout.com: Software Architecture

This new category is about the real side of software architecture. Not buzzwords, not picture-perfect diagrams, but the messy, human part of designing systems that actually work.

I’ve spent many years working with different kinds of architectures, from tightly coupled monoliths to sprawling microservice landscapes. Along the way, I’ve learned that architecture isn’t about doing things the “right” way, it’s about understanding why you’re doing them in the first place, and how they fit into your team and product’s reality.

Note that this applies both to enterprise systems as well as open-source initiatives.

What software architecture actually is

Software architecture is the set of decisions that define how a system behaves, evolves, and is understood over time. It’s not the framework, not the diagram, and not the shiny technology you picked. It’s the reasoning behind those things, and how they hold up when the system changes.

If software design is about the details, like how classes interact, how APIs are structured, and how code is organized, architecture is about direction. It’s the compass that helps the team navigate change without losing its bearings.

Good architecture provides just enough structure for a team to move fast, without locking them into a corner. It’s the scaffolding around the code, the shared mental model that keeps complexity in check.

But architecture doesn’t exist in isolation. The same design that works beautifully in one place can fail completely in another, which brings us to the one factor most people overlook: context.

Why context always matters

There’s no single best architecture, only architectures that make sense in a specific context. A perfect design on paper might fail completely when you drop it into the wrong environment. The right architecture depends on your product stage, your constraints, and, most importantly, your team.

At Frontliners, we’ve seen this up close. We took on the task of replacing a monolith that had been in production for more than 25 years. It was deeply intertwined with the business, used everywhere, and full of edge cases that only existed because of decades of real-world use. To make matters worse, these edge cases are often only known to some users, others just “follow what they’ve been taught”.

We didn’t have a large or deeply experienced team at the time, but we did have a strong sense of purpose. We knew we wanted to modernize, to move toward something distributed and scalable, but we also knew we couldn’t do it all at once. That meant making tough decisions, again and again, weighing what we could achieve now against what would still make sense later. This is especially true from a technical perspective as during this tumultuous time we’ve switched both CPO and CEO multiple times. They all offered new and shiny things, and tech was left trying to realize them within increasingly shorter timelines.

We focused on the people we had, their strengths, their limits, their growth potential, and built an architecture that could grow with them. It wasn’t perfect, but it gave us some momentum, and that momentum led us to something sustainable.

Still, even with the best intentions, it’s easy to lose perspective once you’ve found a path that seems to work. The next challenge we faced wasn’t technical at all, it was cultural.

When good principles turn into dogma

During our early microservices phase, we fell into a trap many teams do. We started creating “rules” that sounded right, but didn’t always fit reality. Things like “you can only do X” or “you can NEVER depend on another service”. And whilst I, as the architect or CTO, never called these “rules” (rather: guidelines), when they were communicated between layers of old <> new developers they often turned into “rules”. Rules like that can be comforting because they feel like control, but they’re often just fear in disguise. Take something like route calculation. When you have hundreds of parameters and thousands of routes, you can’t pre-compute every possible scenario. Sometimes you need to depend on another service, and that’s fine. Architecture should adapt to problems, not deny their existence.

The problem wasn’t the technology, it mostly never is, it was the mindset. Without a clear product goal, people cling to certainty. Those “rules” gave us a sense of safety, but they also made change harder. We eventually had to tear those walls down and rebuild our way of thinking, together.

That rebuilding forced us to look inward and ask hard questions about who we were as a team. Because architecture isn’t just about systems, it’s about people.

Architecture and team maturity: balancing trade-offs

Every architecture exists in the shadow of the team that builds it. A highly mature, cross-functional team can handle complexity. A newer or smaller team cannot, no matter how good the intentions are.

If your architecture outpaces your team’s ability to understand or maintain it, it’s not a good architecture for you. Progress sometimes means taking two steps forward and one step back. You might accept a short-term compromise, a “bad” thing, to enable the next leap forward.

And that’s okay. Because architecture isn’t a competition, it’s a conversation. The best systems evolve through collaboration, not commandments. Let the team make mistakes, learn, and recover. If you enforce every decision from above, you’ll gain consistency, but lose creativity and ownership.

Every engineer should be part of that conversation. They don’t have to think about architecture every day, but they should care about it. Curiosity and challenge keep architecture alive.

Yet even with a healthy mindset, time changes everything. The longer a team works in the same system, the easier it becomes to stop questioning it.

Choice blindness and the value of fresh eyes

As teams settle into a certain way of working, patterns start to feel “normal.” Pain points fade into the background, awkward workarounds become invisible (even desired!). Over time, everyone forgets that things could be different. I call this choice blindness, when familiarity blinds you to your own design decisions.

That’s why new hires are so valuable. They don’t carry the same assumptions. They look at your system and ask the uncomfortable questions: “Why does this work like that?” “Is that rule still needed?” “Has the context changed since we made that decision?”

Sometimes those questions sting. But they’re essential, because context does change. Teams evolve, products evolve, constraints evolve, and what once was a good choice might be holding you back now.

Helping teams see those patterns and guide that evolution is what good architects do best.

The role of the architect

An architect’s job therefore isn’t to dictate (though they are often given that power), it’s to clarify. They connect the big picture to the team’s day-to-day, make trade-offs explicit, and keep people aligned. The most valuable skill an architect has isn’t technical, it’s adaptability.

A good architecture isn’t static. It bends with the product, the people, and the business. And that adaptability only exists when the team is part of the conversation. Architecture that’s understood by everyone lasts longer, because it belongs to everyone.

And that shared ownership matters, because architecture isn’t something you finish, it’s something you continually shape.

Architecture is ongoing, not final

Architecture is never done. It’s a living process that shifts with each decision, sprint, and release. That constant evolution can look chaotic from the outside. To stakeholders or customers, it might even seem like we’re changing direction all the time.

But that’s exactly what makes a system resilient, it changes with its context. You can’t plan your way to perfection, you can only evolve your way there.

The key is communication. When we explain why we’re making architectural changes, and how they serve the product long-term, that “technical mess” starts to look like healthy adaptation. A system that never changes is a dead one.

That’s something we learned first-hand. At Frontliners, we saw how both people and systems evolve, and how one without the other simply doesn’t work.

Wrapping up

At Frontliners, we started with a thirty-year-old monolith and a small, still-growing team. We worked through complexity, limitations, and doubt. We made decisions that weren’t perfect, but they kept us moving.

Now, we have the right people and the right balance. We’re building something sustainable and great. What helped us wasn’t just the technical architecture, but its ability to change with its context.

That, to me, is what software architecture really is.

In the next article, we’ll look at how these ideas translate into structure, comparing monoliths, distributed monoliths, and microservices, and when each one actually makes sense in the real world.

A Hidden Life of Pain, Sorrow, Misery and Rampant Emotions

Sun, 04 May 2025 00:00:00 GMT

⚠️ Content Warning
This blog contains themes of death, grief, mental health, and emotional trauma.
It is raw, honest, and may be deeply triggering for some.
Please read with care and stop if it becomes too heavy. You're not alone.

A Hidden Life of Pain, Sorrow, Misery and Rampant Emotions

Some kinds of missing are temporary. A friend you haven’t seen in a while. A conversation left unfinished. A plan postponed.

But then there’s another kind.

The kind that settles into your bones. The kind that haunts you during the quiet hours. The kind that never finds a conclusion.

That kind is not absence. It’s gone. Irrevocable, final, cruel. A door slammed shut that you still find yourself knocking on, long after you’ve forgotten why you started.

It never opens. And still, you try the handle every now and then.

In the background of every moment

Losing a loved one doesn’t stay in one room of your house though, it isn’t one door that is locked. It leaks under doors and through cracks. You find it in the most unexpected places.

You’re sipping coffee with someone, and they mention a trip they took. Suddenly, you’re remembering the trip you never took. The one you’d planned. The one that cancer took from you.

They talk about a shared playlist. You remember the song you danced to in the rain, soaking wet but laughing anyway. The same song you now skip every time it comes on because it hurts too much

Someone laughs about chipped mugs. You think of the one she cracked and called “vintage,” and how it’s still at the back of your cabinet. Untouched. Sacred.

It’s like walking through a museum where only you know what’s behind the glass.

They talk, they laugh, the world goes on,
Yet I sit with shadows, from dusk to dawn.
A smile here hides a silent scream,
A haunted heart, a broken dream.

These are what I call mind pops. Short for Involuntary Autobiographical Memory Chains. I wrote a journal-like book once, and later a smaller one. I wanted to share them. I tried. But I was never brave enough to.

Mind pops come uninvited. And they come hard. Here are some examples from my book:

The eyeliner she never quite got even.
The way she tucked me in, thinking I was asleep.
The fake wedding we planned just to see what colors we’d choose.
Your love for Twilight. And the moment we realized, quietly, you’d never get to read the next book—if there ever was one.
The mole you’ve had removed and I pretended not to notice. I did miss it so.
The mixture of guilt, remorse and happiness in your eyes as conscious life ebbed away.

They’re not just memories. They’re grenades. Quiet ones. You never know when they’ll detonate. Nor whether they will ruin your entire day.

A flicker in time, unbidden and wild,
A whisper of laughter, a memory smiled.
Then tears arrive with no request,
The heart’s old wounds laid bare, unblessed.

Unfair doesn’t even begin to cover it

We grow up believing that the world has some kind of order to it. That bad things happen, sure—but not this bad. Not this unfair. You lose someone who made life feel infinite. You watch them disappear into the haze of hospital lights and soft-spoken specialists.

Cancer doesn’t care if you’re young. Or kind.
Or if you had a hundred plans left. It just takes.
And keeps taking.

And you’re left with jewelry in a drawer. Her necklace still smells like her.
Sometimes I take it out. I hold it. Smell it. Let it transport me. I don’t even know if I should still be doing that. But it’s a thread. One of the last ones.

This necklace, cold, rests in my hand,
The last soft echo of a silent land.
I breathe her in, though she’s not near,
A ghost in scent, in touch, in tear.

The masks we wear

For a long time, no one knew. Most don’t, still.

People see the version of you they expect: Smiling. Joking. Coping. Existing.
But behind the eyes, you’re unraveling. Constantly. Because grief doesn’t pause.
Not really.

Sharing this is hard. Not because I don’t want to, but because it makes you so vulnerable. Because as you’re writing or speaking, memories pop up uninvited. They derail your words. They choke your sentences. They just pop in… bastards…

And even when you manage to share, others often don’t understand. They compare. They say things like “I lost someone too,” but it’s different. This kind of grief? This kind of loss? It’s heavier. It cuts deeper. And hearing it treated like it’s the same as all the rest—that makes you angry.

Here’s something I once wrote, when I thought no one would ever read it:

I’m surrounded by many figures, but still alone…
No one sees me, nor the pain behind my mask.
They see the smiling, happy guy I show them.
You can’t blame them though, how can I let them see the darkness in my heart?
It would scare them, for it has been torn apart.

I didn’t want to scare people. Or burden them. So I said nothing.
For years.

Strange rituals of survival

Grief makes you do strange things too..
One of my telltale behaviors is that I start eating, lots… but there are weirder ones.
More unexpected ones…

I’ve spoken out loud to pictures. Imagined new conversations. Tried to conjure her in dreams. I’ve spent hours crafting memories that never happened, just to feel like I got one more day. I name items, characters, and other things in games after her. If I eat alone, at a restaurant, I order what she would’ve. (who do I kid, I sometimes do it with company too…)

And I wrote a song. It’s unfinished though—like she was.

🎵 (Refrain)
A strand of blonde hair, as delicate as sun,
A mystery found, in silence it’s spun.
But memory, like a fleeting breeze,
Fades away as a haze, in the time we seize.

🎶(Verse)
Days pass like rustling leaves,
Stories fade, like quiet thieves.
How swiftly memory slips away,
Like an old melody, lost in the fray.

A mysterious gate, in the labyrinth of the mind,
Locked and hidden, what secrets behind?

As I’m writing this blog however, I have a sudden urge to add sound to the lyrics, it’ll still be unfinished, but a little farther along. I used both a piano and a viola to give it a more interesting sound than just the piano.

Your browser does not support the audio element.
score | midi | flac

The final moments and the words that stayed

There are memories too hard to sit with for long. But they come back, as clear as yesterday.

The quiet nights filled with endless conversation. Plans made, even though we both knew we’d never see them through. Fantasies of places we’d never travel, children we’d never have, shows we’d never finish.

She once told me something that still echoes today:

“You have to grieve me, for a while… and then you move on.
You find someone else to love. Someone who makes you feel something again.
You deserve that.”

I shook my head. “I can’t. I won’t. Not because I don’t want to feel again. But because abstinence, to me, feels like remembering you as you are. Eternal.”
We both cried after that. For different reasons, I think.

My words weren’t true though

But life went on. And I did try. I did have relations after her. And every single one felt… off. Wrong. Like stepping into a life that wasn’t meant to fit me anymore.

They weren’t bad people. In fact, I think some of them really tried to care. But the weight of comparison was impossible to escape. The shadow she left was too wide.

Sometimes I’m mad at myself for trying. Other times, I’m mad at myself for failing to let them in. And always,…always,.. I feel like I betrayed her just a little.

I know grief isn’t rational. It doesn’t follow clean lines.
And maybe that’s what hurts most of all—knowing I tried, and still came back to the same place.
Alone, but full of someone who isn’t here.

One day… maybe.

Why I’m finally speaking

It’s taken me over a decade to find the strength to say any of this. And maybe the only reason I can now… is her.

But also—my mother.
She’s terminally ill. And while it’s “natural” for a parent to go before their child, it’s still excruciating.
We don’t know how long she’s got left, but I know we’ll make the best of it. For both of us.

I do find myself ashamed to admit that it hasn’t hit me quite the same way. But it’s not lesser. It’s just different. A different wound on the same, torn, body.
And this time, I’m watching it happen while knowing exactly what’s coming. And I’m scared.
For everyone.

I think often of my brothers. For them, this may be their first real loss. Their first brush with the permanence of death.
And it breaks me in new ways to imagine them feeling what I felt, without ever seeing it coming. I’ll try to be there for them, as you do.

But it’s at least part of why I’m writing now. That’s why I’m finally saying all this.
Because pain grows in the dark.

In a moment of foolish bravery, I once planned to publish my journaled thoughts. That book. That attempt to show what this kind of loss looks like.
I didn’t. I couldn’t.

But maybe I can share this blog.

On those who want(ed) to leave

I need to say something with a very serious tone now, and I do hope you read it gently and thoroughly:

I’ve met people over the years who’ve carried this same grief. And some of them have found themselves teetering.

On the edge.

Wondering if there’s peace on the other side of absence.
Let me say this clearly: I have never thought of taking that road for myself. If anything, I would ask for eternal life. Even with the grief.
But if you are someone who has stood on that edge:

Don’t. Please.

The pain is real. The grief is heavy. But life—even broken life—has light in it still. And sometimes, all you need is one person willing to sit with you in the dark.

Let me be that person for a moment. Just long enough to remind you: the door might be closed, but the room isn’t empty.

If you are on the edge. Seek help. ^{(dutch | english)} Whatever you do, don’t pass on the pain.

What help looks like

Over the past two years, I’ve slowly—achingly—learned to live with it. And honestly, I’m happier now than I’ve ever been since.

I’ve had friends. Real ones. Some with the training to guide me through the murk. Others who simply sat there while I unraveled, trying their best.

They didn’t try to fix me. They didn’t rush me. They just showed up. They had always been there, I just didn’t allow them to be there for me before.

I’ve learned that grief isn’t a wall to climb or a puzzle to solve. It’s a landscape.

You don’t conquer it.
You walk it. One aching step at a time.
And if you’re lucky… you don’t walk it alone.

To those who held me together

There’s a quiet kind of heroism in being the one who stays. Not the one who fixes. Not the one with the right words. Just… the one who stays.

To those of you who sat with me in silence, who didn’t flinch at the weight of my words, who let me cry without needing to understand why. thank you.

To those who asked how I was and meant it. Who asked again when I lied the first time. Who sent messages I never responded to, but always read. Thank you.

To those who reminded me that it was okay to laugh. To love again. To mess up. To heal slow. Thank you.

And to the ones who didn’t even know they helped— you did.

You all held the thread when I couldn’t anymore. And maybe that’s the reason I’m still happy… Still holding her memory, but not drowning in it.

Before I finally sign off

This entire blog was something I never thought I could write. And what comes next,this next part, is something I swore I never would share.

But here we are.

What follows is the foreword and some pages from the book I wrote years ago. A raw collection of thoughts, pain, patterns, and memory. I’ve always wanted to share it, and I’ve always been too scared. Too protective. Too convinced no one would understand.

My grief has changed, not disappeared, never that. But shifted. And time has a way of loosening things you once held too tight. Of showing you that even pain deserves to breathe.

So this is me, still afraid—but doing it anyway.
This is me, defiant in the face of silence.
This is me, opening the door just a little wider.

What comes next is the truth as I once wrote it, in a different voice, at a different time (2021-2022)—but still very much me.

A Peek into the Pain and Misery

The foreword to a book I once wrote titled “A Peek into the Pain and Misery”

We, humans, perceive the vast majority of other humans we encounter as bystanders, observers and otherwise side characters in our own lives. Yet, each and every one of them has a life of comparable complexity to our own. Realizing that for the first time is an emotion called “sonder”.

This book will give you a small glimpse into my life. By the end, you might find that I’m more complex than you’ve thought till now. That there’s another side to the happy, carefree guy I show the world for the majority of my time.

Truth is, that life hasn’t always been easy. In fact, I’d say that after the turning point my life has never felt easy anymore. In fact, for at least 10 years now I’ve been living with prodigious amounts of a single emotion: bereavement.

And though I’ve tried not to let the bereavement take over my life, or in fact make it known to others. it’s always been there. I’ve effectively come to think of it as a zit. Constantly there, constantly annoying and once it’s gone away it’s only a matter of time before it shows up again somewhere else on your body.

Ten years is truly a long time. In it, I’ve tried countless things to forget my sorrow. I’ve written songs and poetry. I’ve indulged in life’s greatest things. I’ve thrown myself into the pursuit of knowledge and I’ve even tried to seclude myself, thinking that I’d become a burden to the world. However, In trying to cope with it I’ve also made many mistakes. I’ve experienced more loss, extirpated meaningful connections and, to my deepest regret, hurt others. And even now, 10 full years later I’m still burdened by the same emotions, though now compounded by the feelings and mistakes of the past 10 years.

Furthermore, During these 10 years, on nearly every day I’ve been haunted by something I’ve come to know as Involuntary Autobiographical Memory Chains, or as I now like to call them: mind pops. Mind pops are sudden memory flashes, seemingly triggered at random, about things I’ve experienced prior. These mind pops never seem to go away and can alter my mood for the rest of the day with the emotions they invoke.

I’ve actually been able to capture the entire story of the past 10 years in a book, of sorts, titled “a hidden life of pain, sorrow, misery and rampant emotions”. And though I’ve always wanted to share the entire story, I’ve never been strong enough to do so. Not 10 years ago, not now, and maybe not ever.

But… The urge to share… something… is still there. So on the 1st of January 2021, I set out on a challenge. I wanted to, at least, share a part of the story. Not one from a distant past though, one of constant remembrance. One of pain and pleasure. One I’ve lived for every day of the entire year. One I’ll live for every day for many years to come.

Every page henceforth will contain one “mind pop” that I’ve had on the date listed. So turn over this page and join me on my struggle from January 1st through December 31st of 2021.

A postscript

If you’ve read this far—thank you. That alone means more than you know.

- Rick, mastermindzh, lycan, xxxroosjexxx, and other alias’ you might know me from.

Infrastructure as Code - Simplifying My Homelab Setup with Portainer GitOps

Sat, 03 May 2025 00:00:00 GMT

After writing my last blog post about using Infrastructure as Code for uptime monitoring with UptimeKuma, I felt inspired to dive deeper into my broader homelab story.

From Complexity to Clarity

When I started building my homelab, it was all about experimenting and having fun. Kubernetes, Ansible, HA clusters spread across multiple nodes… I dove into everything headfirst. The concept of Infrastructure as Code (IaC), something I fell deeply in love with thanks to Kubernetes, quickly became the backbone of my ideal setup.

Yet, as much as I enjoyed the complexity and elegance Kubernetes brought to my experiments, I soon realized something pretty obvious: when it comes to crucial services like Jellyfin for movies or Home Assistant for managing my smart home, the last thing I needed was an over-engineered, highly available Kubernetes cluster requiring constant babysitting.

The Before Times

It reminded me of simpler times, the “before times”, before Docker and Kubernetes even existed. Back then, my homelab consisted of multiple Ubuntu servers, each handling different parts of my setup, along with a single, reliable main server named “Erebus,” using Greyhole for redundancy. Even when Docker came along (and later Kubernetes), I adopted them immediately for non-critical, experimental projects, but crucial services always remained comfortably on one stable server. After all, Kubernetes-level scalability was never really necessary for my home media or automation and I could still use all the benefits of Docker.

Hades, the Storage Behemoth

Today, this philosophy still holds true. My critical services run smoothly on a single Unraid server, my trusted main storage server named “Hades.” On Hades, it’s a pleasantly chaotic mix of Unraid’s Docker plugin, community apps, and various Docker Compose files. Each container has decentralized configuration variables, with updates effortlessly handled by Unraid’s auto-update plugin.

Here’s the ironic twist: despite having Kubernetes nodes running non-critical, playful stuff, all my serious storage still ends up on Hades. Honestly, nothing else compares. Hades dwarfs every other node in my network storage-wise, think around 200TB of free space at any given time. Even Kubernetes relies heavily on Hades for its back-end storage.

Simplifying with Portainer GitOps

Still, this fragmented setup felt manual and messy. I missed the simplicity and clarity of Infrastructure as Code. Thankfully, Portainer’s Ops feature came to my rescue, allowing me to blend the massive storage and ease of use from Unraid with structured IaC management. To fully leverage this, I registered for 3 free nodes of Portainer Business Edition, unlocking their powerful GitOps feature.

Portainer’s GitOps polls my private Git repository for updates to Docker Compose files at regular intervals, automatically deploying any changes. It seamlessly integrates with Docker Hub, pulling the latest container images and ensuring my stacks remain updated without manual intervention. This automation keeps my services fresh and secure with minimal effort.

The UptimeKuma example from my last blog is one such stack I recently migrated to this new, cleaner workflow.

The Best of Both Worlds

Now, I have the best of both worlds—a robust and neatly structured setup embracing Infrastructure as Code without sacrificing ease of use or storage headaches. In other words, I could keep the things I liked best about my Kubernetes (the predictability and IaC parts) and got rid of its worst (multiple nodes, manual updates).

On to the next adventure!

Infrastructure as code - Service uptime monitoring

Sat, 18 Jan 2025 00:00:00 GMT

If there’s one thing I’ve always embraced, it’s the philosophy of working smarter, not harder. Infrastructure as Code (IaC) has been one of the cornerstones of my career. a perfect blend of laziness and the pursuit of predictability. From my earliest days experimenting with Docker back in 2014 to leading a platform team at my current company, IaC has proven invaluable for simplifying complexity, ensuring reproducibility, and enabling automation.

Below you’ll find my journey and why I think every engineer should embrace it, eventually walking you through one of the latest automations I set up at home: Automatically creating uptime monitors in UptimeKuma based on Docker labels.

The Early Days: Discovering Docker

Back in 2014, I stumbled upon Docker, and it was a game-changer. At the time, I was frustrated by the manual and error-prone process of setting up environments for my code (especially on other people’s machines…). Docker offered a way to reliably recreate these environments with simple, declarative configuration files. Suddenly, I could spin up a development environment in minutes and be confident it would work exactly the same on another machine.

In 2017, I taught several university classes on Docker, emphasizing the importance of reproducibility. My students learned how to containerize applications and ensure their work could be shared and run anywhere, regardless of underlying infrastructure. Reproducibility wasn’t just a technical advantage; it became a mindset I encouraged in every project.

And yes.. for those that know me well, I did go overboard in some of the details and even ended up explaining the entire Copy-On-Write (COW) nature of the Docker filesystem in those classes… But ah well, people learned a lot :P.

Home Servers: The Personal Lab

My passion for IaC extended to my home servers. By 2017, nearly everything I ran at home was Docker-based. I created dozens of bash scripts and system services to orchestrate my personal infrastructure. Whether it was media servers, backup systems, or development environments, everything was automated and version-controlled. Even my own computers mostly became IaC based as I figured out that part of the Linux community was saving their setups (and install instructions, usually) in so called “dotfiles”. To this day you can still find my setup (and its changes when I switch machine) in my Dotfiles on Github.

All in all, my home setup became a sandbox for testing new ideas and tools, many of which eventually found their way into my professional work as well.

Scaling IaC in the Workplace

At my current company, I introduced Docker about eight years ago. It was a gradual process, but within a year, we had our first Kubernetes cluster running. This transition wasn’t just about adopting new tools; it was about embedding the principles of IaC into the organization’s culture.

Over time, I spearheaded the creation of a dedicated platform team. With an architect/Product Owner and four DevOps engineers, this team took IaC to the next level. They implemented robust CI/CD pipelines, infrastructure monitoring, and scalable deployment patterns. Some of these practices mirrored what I had done at home, while others were tailored to the unique needs of the business. The result was a resilient and predictable infrastructure that supports rapid development and deployment.

Setting Up Uptime Monitoring with UptimeKuma

One of the most satisfying aspects of IaC is the ability to automate even the smallest tasks. Take uptime monitoring, for example. At home and at work, ensuring that services are available is critical. Recently, I’ve been using my old UptimeKuma instance, a self-hosted monitoring tool that’s as powerful as it is user-friendly, a lot more after introducing it to some friends (who started homelabbing) and at work.

Setting up UptimeKuma is straightforward, you can simply use our old friend Docker:

docker run -d --name uptime-kuma -v ./data/uptimekuma:/app/data -p 3001:3001 louislam/uptime-kuma

And access the Dashboard by navigating to http://localhost:3001 to configure your monitors. But that isn’t automatic enough for me, I like to put my things in compose files for home usage.

Automating uptime monitors with AutoKuma

AutoKuma allows us to set labels on our Docker containers that will then automatically generate monitors in UptimeKuma.

One of my containers (UptimeKuma actually) has the following labels attached:

labels:
  kuma.monitoring.group.name: "Monitoring"
  kuma.uptime_kuma.http.parent_name: "monitoring"
  kuma.uptime_kuma.http.name: "Kuma status monitoring"
  kuma.uptime_kuma.http.url: "http://${HOST_IP}:3001"

This actually does 2 things:

creates a group with the key/id monitoring and the name Monitoring
Adds a monitor with the key/id uptime_kuma to UptimeKuma with the type http, name Kuma status monitoring, and url http://${HOST_IP}:3001

Adding these labels, whilst AutoKuma is running and configured to pick up labels starting with kuma is enough for monitors to show up (after restarting the containers). All in all, my docker-compose.yml file for both UptimeKuma and AutoKuma now looks like this:

services:
  autokuma:
    image: ghcr.io/bigboot/autokuma:master
    restart: unless-stopped
    environment:
      AUTOKUMA__KUMA__URL: http://${HOST_IP}:3001
      AUTOKUMA__KUMA__USERNAME: ${KUMA_USERNAME}
      AUTOKUMA__KUMA__PASSWORD: ${KUMA_PASSWORD}
      AUTOKUMA__TAG_NAME: AutoKuma
      AUTOKUMA__TAG_COLOR: "#42C0FB"
      AUTOKUMA__DEFAULT_SETTINGS: |-
        docker.docker_container: {{container_name}}
        http.max_redirects: 10
        *.max_retries: 3
      AUTOKUMA__DOCKER__LABEL_PREFIX: kuma
      AUTOKUMA__MIGRATE: true
    volumes:
      - ${APP_DATA}/autokuma:/data
      - ${DOCKER_SOCKET}:/var/run/docker.sock
    depends_on:
      - kuma

  kuma:
    image: louislam/uptime-kuma:1
    volumes:
      - ${APP_DATA}/uptimekuma:/app/data
      - ${DOCKER_SOCKET}:/var/run/docker.sock
    ports:
      - 3001:3001
    restart: unless-stopped
    labels:
      kuma.monitoring.group.name: "Monitoring"
      kuma.uptime_kuma.http.parent_name: "monitoring"
      kuma.uptime_kuma.http.name: "Kuma status monitoring"
      kuma.uptime_kuma.http.url: "http://${HOST_IP}:3001"

Problems… the compose file doesn’t work :O

If you’d try to run the compose file in the previous chapter, even after replacing all the variables, things likely still won’t work. This is because AutoKuma relies on the credentials for UptimeKuma which we setup during the initial launch of UptimeKuma. Unfortunately, that account is the only account we can currently set up in UptimeKuma since it doesn’t have user management. And yes, that means we have to use these same credentials to get AutoKuma to work, which also prohibits us from enabling MFA.

Anyway, after setting up the kuma account, simply adjust both the KUMA_USERNAME and KUMA_PASSWORD variables and restart the containers. After doing so we should see the monitor appear in UptimeKuma:

neat!

The Future of IaC

Infrastructure as Code isn’t just a technical approach; it’s a philosophy that prioritizes automation, reproducibility, and simplicity. Whether you’re managing a home lab or a global platform, IaC provides the tools and practices to build resilient systems with minimal effort.

For me, IaC has been a journey of continuous learning and experimentation. From Docker to Kubernetes, from bash scripts to dedicated platform teams, the principles remain the same: automate everything, document everything, and embrace the predictability that code brings to infrastructure.

If you haven’t already, give UptimeKuma and AutoKuma a try. Their combination is an excellent example of how IaC can simplify even the most mundane tasks, leaving you more time to focus on what really matters… or just to be a little lazier.

LegoLeague - Battling it out with little robots made from Lego

Mon, 18 Sep 2023 00:00:00 GMT

The 2023 LegoLeague battle board

A little before the start of the summer period my colleague Jan-Kees came up to me and proposed the idea of an internal LegoLeague. After a little bit of back-and-forth about what LegoLeague exactly was I agreed it would be a great idea to bring some colleagues together and have a little fun during work hours. Pretty much right away excitement got the better of us and we started drafting a plan… when? where? how? prizes? teams!?? A lot. But First…

What is LegoLeague?

LegoLeague is an event where teams come together to build, program and test a Lego Spike robot to perform autonomous tasks. The tasks range from collecting “power cores”, pushing or pulling levers, to dropping Lego blocks in a hopper to get some bonus points.

All of this is accompanied by a theme, in our case energy production, which adds a layer of fun to solving these challenges. The video below shows a teaser of 2022-2023’s “SuperPowered” event, the very thing we based our LegoLeague on:

For each task a team completes successfully they will score points. Teams get 2 minutes and 30 seconds (2:30) to score as many points as they can. The video below outlines some of the challenges the teams have available to them with a little explanation of how they function and even some info on the scores. This video is especially useful for teams that don’t want to peruse the entire rule book but still wanted to figure out an optimal strategy.

To build the robot each team gets a Lego Spike Prime set and nothing more! In the original event, teams had access to a little more Lego in various kits but for our little event at INFORIT, we were only able to source the basic kits in time. Having said that, we wouldn’t have been able to get our hands on any of the kits at all if it wasn’t for the “PReT” organization in Weesp. Thanks guys!

A Lego spike set laid out on display

As for the programming… Teams are free to use either ”scratch” or “Python” to program their robots. Scratch being much easier for those who don’t spend all day behind their computer working on software development whilst Python allows for more flexibility in what can be achieved. Spoiler alert: all teams chose scratch, it’s much more accessible for non-developers!

The competition at INFORIT

So yeah, we now know what a LegoLeague is and we have the approval to host the competition at INFORIT, now what? Well, Jan-Kees got to work building the arena and setting it up and I set out to gather team members. Given that it was the summer break I had to hustle to get teams together but eventually we managed to create some beautiful teams.

Fortunately, INFORIT values employee happiness, autonomy and continuous improvement. As such I was able to offer the 2 Friday afternoons free of day-to-day work, to work on the robot whilst providing catering, music, and the like to the teams whilst they are participating!

In the end, we were successful in creating 4 teams of at least 3 people who were all mixed with developers and non-developers to keep the balance fair. Meanwhile, Jan-Kees also finished building the arena which you can see in the header of this blog or the pictures in the picture section.

Pictures

Of course, we’ve taken loads of pictures during the event but those are mostly for internal use. Below I’ve included some pictures of the event that are all cleared for social media distribution with a little description of what’s happening in each picture to, hopefully, convey some of the vibes of the event itself.

one of the competing robots

A contestant calculating exactly where to go. Such a pathfinder!

code -> test -> repeat. Soooo important

This team might have overcomplicated things a tad. Looks impressive though.

Winners, losers, etc

In the end, this is still a competition. People competed hard to come out on top and 1 team did so successfully. Congratulations guys!

All of our winners! (excluding myself 😢)

Nah, we are all winners in the end

Having 4 teams gave us the unique opportunity to reward them all with something nice. The 3 teams that got podium places all got their medals and their rewards and the team in fourth place got their consolation prize.

Luckily though, we’re all winners, even the Jury members and the organizers. This event has brought us all closer together and improved the connection between our colleagues. I, personally, had a blast and would love to do similar things in the future.

That’s it again, thanks for reading and see you next time!

Flashing the LSI-9211 to IT mode using an EFI shell

Wed, 12 Apr 2023 00:00:00 GMT

I’ve been building a new storage-oriented server for a while now and have yet again decided to go with (3 +1 backup) LSI 9211 raid controllers.

The reason I keep going for these specific raid controllers is quite simple. The card can theoretically support (8x500MB) 4GB/s in throughput divided over 8 drives (2x SFF-8087) which is near the maximum for consumer hard drives.

Using 3 of these will allow me to use 24 drives in a single chassis without an expander whilst getting pretty good performance. The fourth card that I got, and flashed, is simply used as a backup or debug card in case of any errors.

Downloading the files

Before we can start flashing we will need to download some files. The table below lists all the files with a brief description of what we’ll use them for.

File	Description
EFI flashing utility	Flashing utility
EFI ROM image	Card ROM file
SAS2008 Firmware	IR and IT firmware for the 9211

Prepping the flash drive

Make sure that you have a flash drive that is 32GB or smaller and that it is formatted as FAT32. Next, we’ll add the following files to the root of the flash drive:

Installer_P20_for_UEFI/sas2flash_efi_ebc_rel/sas2flash.efi
UEFI_BSD_P20/uefi_bsd_rel/Signed/x64sas2.efi
9210-8i_Package_P20_IR_IT_FW_BIOS_for_MSDOS_Windows/sasbios_rel/mptsas2.rom
9210-8i_Package_P20_IR_IT_FW_BIOS_for_MSDOS_Windows/Firmware/HBA_9210_8i_IT/2108it.bin

Flashing with EFI

Because we are going to flash using EFI we don’t really need anything besides the built-in EFI shell from our motherboard. Simply insert the flash drive and go into the BIOS or use the “select boot device” functionality to select the EFI shell:

selecting the EFI shell boot option

After the shell has booted successfully you can use the map command to see all available devices. In the image below you’ll see that my flash drive is listed as fs1:

a list of devices note that `fs1` is my flash drive

To switch into the drive simply enter the drive name on the command line followed by a semicolon:

fs1:

Next, we’ll determine that our adapters are available and we will simultaneously take note of their identifier, we can do so by issuing the following command:

sas2flash.efi -listall

a list of adapters, I flash them one at a time so I only have card '0' available to me

Next, we can empty the current firmware with the following command:

sas2flash.efi -o -e 6

bye, bye firmware 😢

Now we’ll simply flash the new (IT-mode) firmware on it with the following command:

sas2flash.efi -o -f 2108it.bin -b mptsas2.rom

1 card down, 3 more to go!

Wait patiently until the operations have completed. Note that it might seem like the steps repeat, just wait until it is completely done and exits back to the shell by itself.

Multiple cards and conclusion

You can rerun the commands for different cards (by adding -c x where x is the card number from listall) or by simply physically switching the current one out for a different one.

That’s all there is too it, if you need any help feel free to hit me up!

Uncovering a new hobby that might just take over my entire house

Tue, 14 Feb 2023 00:00:00 GMT

The problem

Even though my house is fully furnished, one could say it’s quite light on decorations. You see, I’m not really one for many of the popular decoration pieces like candles, picture frames, animal skins, or decorative statues.

The only thing I really like is stone and woodwork. This means that I have enough decorations to fill the downstairs, kind off. You see, some of the decorations that I also like are plants (mainly because they go in large stone pots) but I’m not great at keeping them alive.

(Un)fortunately, there is more to my house than just the downstairs. There are 2 more floors that are, once again, furnished but not really decorated. The only room that is pretty well decorated is my office, which has various little nerdy things on little black shelves and 12 beautiful Displates.

One of the main problem areas is my bedroom. I don’t have any entertainment in my bedroom (by choice.. it’s a place to sleep, not watch telly!) but I do have some FJallbo shelving units. Of course, those shelves are mostly empty… they are only adorned by some books, clothes and a few stuffed animals.

The solution, or so I thought

Pretty little orchids!

I don’t know when exactly it happened but a few months ago I stumbled across “Lego plants” on Youtube. Plants that look quite good and don’t die. The first plant I got was the little bonsai and the second one were the orchids. After that, I quickly picked up the cacti and bouquet.

And as you can probably figure out by now… a new hobby was born… Lego. Lego allows me to fill up the empty space with whatever I want whilst keeping things interesting and giving me something to do to boot.

Christmas time’s a coming

After building a few smaller items I felt ready to tackle a “project”, one that would allow me to play with Legos and at the same time get a few friends and family members off my back about not having Christmas decorations. That’s right. I set out to build a Christmas village from lego.

I emptied a shelf, picked up some white base plates, procured a string of small Christmas lights, and bought some Christmassy-looking Lego sets:

The result of a few days of work and the Christmas village has been completed:

Happy little Christmas village

A new room to build Lego

Now that the Lego virus has taken hold of me I am quickly growing tired of needing to move stuff off my dining table just to build some Legos. Likewise, I dislike having to move half-finished Lego builds just to use my dining table. It was time to upgrade my spare bedroom with a desk and some other things to build a comfortable space.

I bought the following items:

Which, with some effort, got turned into the following room:

First build of the room, no display or audio yet

A little trolley to hold the new Lego sets

I added a monitor so I could watch some series or listen to music.

Sound from just the monitor won't do so I added a nice JBL speaker

the finished room in use, building the Harry Potter special edition owl

The space I’ve created with this Ikea build works very well for building Legos! The mounted desk light (which I got from Amazon) has also proven to be quite handy. I can move the light however I like, be it to shed some more light on the little booklet with instructions or on the build itself to illuminate some of the darker pieces at night. (brown/black) Turning on both lights (there’s one hidden behind the monitor/speaker) ensures that there’s enough light in the room that I don’t have to turn on the main light which makes the room feel nice and cozy.

More of my creations

I’d like to round off this blog post by showing some of the other builds I’ve since completed:

Saturn V

Shout out to Hedwig! (and Ravenclaw)

The desk felt a bit empty so I added some flowers to spice things up

Two of the succulents in decorative pots with some shells

Adopting conventional commits and standard versions in a git and npm codebase

Mon, 07 Nov 2022 00:00:00 GMT

At my place of work, INFORIT, we are talking about adopting conventional commits as a company standard. However… debating commits is a tricky endeavor, everyone seems to have their own opinion about it and pretty much no one is following even the basic git rules (e.g title + body). It is precisely because of all of this that an earlier proposal about standardizing commit messages brought us nothing.

Now though, a solid 2 years later, our company has grown tremendously! Not just in actual size, but also in knowledge. The developers that were resistant to standardization, hell… even git, have since come around to many of the ideas that were implemented. Especially after seeing their, often new, colleagues thrive with it.

After seeing one of the new hires use conventional commits and their team being fine with it I decided to take the plunge and submit a proposal for it myself. Though the proposal only covers “conventional commits” and generating changelog entries from it I will be explaining the entire process of adopting conventional commits and standard versions in this blog post.

Getting started

Before we can do anything we need a git repository. I’ll be using a hosted git repository (on Github) so I will clone it:

git clone git@github.com:Mastermindzh/conventional-commits-showcase.git

But, if you want to follow along without a hosted repository you can also manually create it yourself:

mkdir conventional-commits-showcase # create a folder with the name "conventional-commits-showcase"
cd conventional-commits-showcase
git init

What exactly are “conventional commits”

”Conventional commits” is a specification that allows developers to write commits that are easily parsable by both developers and systems alike. Or in their own words:

The Conventional Commits specification is a lightweight convention on top of commit messages. It provides an easy set of rules for creating an explicit commit history; which makes it easier to write automated tools on top of. This convention dovetails with SemVer, by describing the features, fixes, and breaking changes made in commit messages.

According to the specifications the commit message should be structured as follows:

<type>[optional scope]: <description>

[optional body]

[optional footer(s)]

The specification also allows us to specify a “footer” with breaking changes and ticket references. A full example, with a breaking change and a fictional ticket number of 95043 would look like this:

feat(api): A test title for a change to the API

I made a few changes to the API. Now I have to write a body.
This is a looooooong body.
Multiple lines :O

BREAKING CHANGE: Well, I have to write something here... so :)
fixes #95043

Let’s get the dependencies installed

First, let’s make sure our repository contains an npm environment. If not, run:

npm init

Now that we have a package.json file we can start installing some dependencies:

npm install --save-dev husky @commitlint/cz-commitlint commitizen commitlint standard-version

That will give us all the dependencies we need during this blog:

Dependency	function
Husky	Allows us to easily manage and execute git hooks
commitlint	Checks whether our commit messages meet the conventional commits specification
@commitlint/cz-commitlint	adapter for commitlint to use commitizen
commitizen	Provides a prompt during commit to fill in necessary information
standard-version	Semver based release management and changelog generator based on conventional commits

Setting up the commitlint and commitizen configuration

First, let’s add a script to the package.json that will do our commit for us:

{
  "scripts": {
    "commit": "git-cz"
  }
}

Now, when we call npm run commit it will actually run git-cz which will start our commitizen prompt. You can try it if you want, it won’t actually do anything but throw an error right now. We can also configure commitizen in the package.json as well, to do so we add commitizen to the package.json’s config object:

{
  "config": {
    "commitizen": {
      "path": "@commitlint/cz-commitlint"
    }
  }
}

If you ran npm run commit before and got an error, you can now do it again and face another, but different, error!

Now we’ll set up the actual commit linting. Start by creating a file commitlint.config.js with the following content:

module.exports = {
  rules: {
    "body-leading-blank": [1, "always"],
    "body-max-line-length": [2, "always", 100],
    "footer-leading-blank": [1, "always"],
    "footer-max-line-length": [2, "always", 100],
    "header-max-length": [2, "always", 100],
    "subject-case": [2, "never", ["start-case", "pascal-case", "upper-case"]],
    "subject-empty": [2, "never"],
    "subject-full-stop": [2, "never", "."],
    "type-case": [2, "always", "lower-case"],
    "type-empty": [2, "never"],
    "type-enum": [2, "always", ["first type", "second type"]],
  },
  prompt: {
    questions: {
      type: {
        description: "This is the first prompt!",
      },
    },
  },
};

As you can see we have configured a whole bunch of things, but there are 2 things that matter for now:

a single “question” in our prompt.
a few types (first and second type)

Staging a few files (git add <file>) and running npm run commit will show us our prompt and a choice between the two types:

output of 'npm run commit'

Now, our current config isn’t very useful, so let’s add some more types and a few additional questions (the defaults):

module.exports = {
  rules: {
    "body-leading-blank": [1, "always"],
    "body-max-line-length": [2, "always", 100],
    "footer-leading-blank": [1, "always"],
    "footer-max-line-length": [2, "always", 100],
    "header-max-length": [2, "always", 100],
    "subject-case": [2, "never", ["start-case", "pascal-case", "upper-case"]],
    "subject-empty": [2, "never"],
    "subject-full-stop": [2, "never", "."],
    "type-case": [2, "always", "lower-case"],
    "type-empty": [2, "never"],
    "type-enum": [
      2,
      "always",
      [
        "build",
        "chore",
        "ci",
        "docs",
        "feat",
        "fix",
        "perf",
        "refactor",
        "revert",
        "style",
        "test",
        "content",
        "blog",
      ],
    ],
  },
  prompt: {
    questions: {
      type: {
        description: "Select the type of change that you're committing",
        enum: {
          content: {
            description: "A new piece of content",
            title: "Content",
            emoji: "✨",
          },
          feat: {
            description: "A new feature",
            title: "Features",
            emoji: "✨",
          },
          fix: {
            description: "A bug fix",
            title: "Bug Fixes",
            emoji: "🐛",
          },
          docs: {
            description: "Documentation only changes",
            title: "Documentation",
            emoji: "📚",
          },
          style: {
            description:
              "Changes that do not affect the meaning of the code (white-space, formatting, missing semi-colons, etc)",
            title: "Styles",
            emoji: "💎",
          },
          refactor: {
            description: "A code change that neither fixes a bug nor adds a feature",
            title: "Code Refactoring",
            emoji: "📦",
          },
          perf: {
            description: "A code change that improves performance",
            title: "Performance Improvements",
            emoji: "🚀",
          },
          test: {
            description: "Adding missing tests or correcting existing tests",
            title: "Tests",
            emoji: "🚨",
          },
          build: {
            description:
              "Changes that affect the build system or external dependencies (example scopes: gulp, broccoli, npm)",
            title: "Builds",
            emoji: "🛠",
          },
          ci: {
            description:
              "Changes to our CI configuration files and scripts (example scopes: Docker, Kubernetes, npm, git)",
            title: "Continuous Integrations",
            emoji: "⚙️",
          },
          chore: {
            description: "Other changes that don't modify src or test files",
            title: "Chores",
            emoji: "♻️",
          },
          revert: {
            description: "Reverts a previous commit",
            title: "Reverts",
            emoji: "🗑",
          },
        },
      },
      scope: {
        description: "What is the scope of this change (e.g. component or file name)",
      },
      subject: {
        description: "Write a short, imperative tense description of the change",
      },
      body: {
        description: "Provide a longer description of the change",
      },
      isBreaking: {
        description: "Are there any breaking changes?",
      },
      breakingBody: {
        description:
          "A BREAKING CHANGE commit requires a body. Please enter a longer description of the commit itself",
      },
      breaking: {
        description: "Describe the breaking changes",
      },
      isIssueAffected: {
        description: "Does this change affect any open issues?",
      },
      issuesBody: {
        description:
          "If issues are closed, the commit requires a body. Please enter a longer description of the commit itself",
      },
      issues: {
        description: 'Add issue references (e.g. "fix #123", "re #123".)',
      },
    },
  },
};

As you can see we have a fair few questions, each with its own description. The one that will be new to most people is “content”, which is a custom type that I added for this very blog. Reenacting our example commit from before will look like this:

Looking good so far!

That’s pretty nice already! Creating our example commit is nice and easy now. Next, let’s add a git hook to stop us from messing up.

Setting up git hooks

Without the git hook, we can simply make a commit that doesn’t adhere to conventional commits:

mkdir test-files && touch test-files/test1
git commit -m "this is not conventional :O"

Yikes! That is not what we want! Let’s quickly revert it:

git reset --soft HEAD~1

Now, let’s add a Husky git hook to combat future commits that are unconventional:

npx husky-init && npm install

This will automatically create the .husky/ directory. Let’s create a file in it called commit-msg and add commitlint:

#!/usr/bin/env sh
. "$(dirname -- "$0")/_/husky.sh"

npx --no-install commitlint --edit "$1"

Let’s try our unconventional commit again:

touch test-files/test2
git add test-files/test2
git commit -m "this is not conventional :O"

nice! It throws an error :)

Great, our commits seem to be working. Let’s commit all the things we have now before we move on, crucially I will be using the regular git commit and providing the conventional commit syntax myself:

git commit -m "ci(git): Added conventional commits and some dependencies"

Releases

Now that we’ve successfully set up conventional commits we want to use those to release our first version and generate a changelog to boot. First, let’s add some npm scripts again:

{
  "scripts": {
    "release": "standard-version",
    "release:minor": "standard-version --release-as minor",
    "release:patch": "standard-version --release-as patch",
    "release:major": "standard-version --release-as major"
  }
}

Now, let’s add a configuration file for standard-version so that it will generate changelog entries. Create a file named .versionrc.json with the following content:

{
  "types": [
    {
      "type": "feat",
      "section": "Features"
    },
    {
      "type": "fix",
      "section": "Bug Fixes"
    },
    {
      "type": "content",
      "section": "New content",
      "hidden": true
    },
    {
      "type": "chore",
      "hidden": true
    },
    {
      "type": "docs",
      "hidden": true
    },
    {
      "type": "style",
      "hidden": true
    },
    {
      "type": "refactor",
      "hidden": true
    },
    {
      "type": "perf",
      "hidden": true
    },
    {
      "type": "test",
      "hidden": true
    }
  ],
  "commitUrlFormat": "https://github.com/mastermindzh/conventional-commits-showcase/commits/{{hash}}",
  "compareUrlFormat": "https://github.com/mastermindzh/conventional-commits-showcase/compare/{{previousTag}}...{{currentTag}}"
}

The first thing we declare is an array of types, for every type we can define a “section” and whether it is hidden or not. All hidden types will not show up in the changelog, the others will be grouped by type and will have a header with the text specified in the “section” field. Let’s try it:

git add .versionrc.json package.json
git commit -m "ci(git): Set up standard-version"

# and let's add a feature
touch test-files/test3
git add test-files/test3
git commit -m "feat: Feature test3"

note: Where I’ve been using git commit -m you can also use npm run commit to get the interactive git prompt. Now, let’s release a major version:

npm run release:major

our first release!

The output is kind enough to tell us how to push our changes to the remote so let’s just do it:

git push --follow-tags origin master

Now let’s have a look at our git log:

we have our own commits + a new release commit

Now, let’s also look at our changelog:

very neat!

That looks great! standard-version automatically added a header for our version, sub-headers for our types and even linked our commits to the remote as configured in .versionrc.json.

Let’s play a little

Now that we have a successful setup we can try a few commits and releases. Let’s start with 2 bugfixes and a new feature:

# our first bugfix
touch test-files/test4
git add test-files/test4
git commit -m "fix: fixed test 4 by doing fancy stuff"

# our second bugfix
touch test-files/test5
git add test-files/test5
git commit -m "fix: test5 was difficult, but we got it"

# and finally a new feature
touch test-files/test6
git add test-files/test6
git commit -m "feat: Added feature number 6! So nice"

Now we’ll release this as a minor, because we added a new feature. Execute

npm run release:minor

our updated changelog

breaking changes

All is well, but let’s introduce a breaking change:

touch test-files/test7
git add test-files/test7
npm run commit

Output:

> conventional-commits-showcase@1.1.0 commit
> git-cz

cz-cli@4.2.5, @commitlint/cz-commitlint@17.2.0

? Select the type of change that you're committing: feat
? What is the scope of this change (e.g. component or file name) (press enter
to skip): (max 96 chars)
 (0)
? Write a short, imperative tense description of the change: (max 96 chars)
 (38) feature 7, with a few breaking changes
? Provide a longer description of the change (press enter to skip):
 We broke some stuff, woopsie
? Are there any breaking changes?: Yes
? Describe the breaking changes:
 api endpoint 'test7' now returns json instead of xml
? Does this change affect any open issues?: No
[master e7f7b0e] feat: feature 7, with a few breaking changes
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 test-files/test7

Which gives us the following changelog:

everyone will see this message, no users will complain, no way!

Conclusion

Phew! That was quite a long post. Hopefully, you’ve managed to set up conventional commits and/or standard-releases to your liking and if not there’s always the comment section. The end-result can be found on Github, the Changelog is especially pretty.

My blog hasn't gotten an update in 5 years, here's why.

Sun, 30 Oct 2022 00:00:00 GMT

I’ve been rebuilding the old ”rickvanlieshout.com” for over 5 years now… But the website isn’t that big. It could be done way faster and it has been done way faster. The thing is, I rewrote it several times, each time with a different design and updated packages.

Changing the theme (round #1)

Shortly after finishing the original website its design already felt stale. Sure, it had some nice colors and I really liked the mobile view (it read beautifully on a smartphone), but it still felt off. Nothing on the website gave me a wow effect, most of it was “ok” and some of it was not to my liking. I never quite put my finger on the why, but it just didn’t feel right.

I did put a lot of time into optimizing and designing the bits I found important, the biggest time sinks were multi-language support and print mode. In the end, I even ended up making a compromise on the multi-language support because I didn’t want to blog in Dutch. One of the things I’m glad I did, however, was working on styling print mode. It wasn’t because I expect many people to print my website or articles, it’s because I expect some of them to use “read-focus” extensions or apps like pocket as I do so often myself. I also suspect the occasional pdf download, most of which I probably did myself on my resume page though.

One of the biggest annoyances that I tried solving with a new theme was dark mode. Hell, I swung completely in the opposite direction and bought the all-dark Hikma theme. I rebuilt the entire site in that theme, but never published. The next chapter will explain why.

Moving to Gatsby

Right around the time I started thinking about uploading the new theme I also found GatsbyJS. Gatsby was in its infancy back then, barely out of beta and a little rough around the edges. Considering I like to live on the edge, software-wise anyway, I decided to go for it. rickvanlieshout.com-hikma-gatsby was born and I worked on it for a few short weeks.

In the end, that also burned out because, again, the theme felt stale.

Upgrading to Gatsby 3 and back to the old theme

I’d had enough! I needed to finish my website and I needed to do it quickly. I picked up the new Gatsby V3, my old theme, and a renewed mental vigor and started coding away. The first prototypes came into existence very quickly and looked fine.

Alas, I burnt out on the project almost as quickly as I had started. One of the reasons was the theme… but there was a more serious reason this time as well.

Server-side rendering, the old theme, and a stubborn software architect

During the rewrite to Gatsby V3 I encountered various SSR issues. The most notable of them all was not being able to save and apply the color scheme without annoying flashes on page loads. I chalked it up to a CSS issue and left it for what it was at first, but I simply couldn’t go live like this.

We just discovered my other biggest flaw. Perfectionism. Because I am responsible for code and product quality at my job and have also heavily emphasized that part of my contributions in any of the open-source projects I’ve worked on, I felt that I couldn’t release something that wasn’t perfect. And as most of you will know, you can chase perfection forever. There is always a flaw, always something to improve.

In my mind, I could fix the quality issues, but I’d still be left with the theme. I also realized I’d end up at the same point I started… Same design, and the same missing features, just a bunch of time later. This wasn’t working either.

Finding a new design and a new starting point

One day I found myself procrastinating on a whole bunch of things by looking at fancy CSS themes. I hadn’t really thought about it, but I’d ended up looking at new portfolios and blog designs. I’d found a few good candidates but didn’t really make a choice yet.

Was I embarking on the journey once again? Or would I give up before I started? Eventually, I decided to organize my thoughts and figure out why things failed before, it roughly came down to:

I can’t go about and make it perfect on the first try
- No translations
- Print mode for all the important pages
- Dark mode can come later
I didn’t like the design, I wanted something black and white which could be reversed for dark mode
I didn’t like the previous efforts of using Gatsby for just the posts and not the static pages
I needed a starting point, no matter how “good” it was, I needed a place to start writing again.
I needed a new website fast because the old one was breaking down (PHP updates caused the API to break)

Eventually, I stumbled upon ”lumen”. A beautiful blog project with amazing typography and a relatively modern Gatsby setup. It wasn’t perfect, it missed a few bells and whistles, but it did tick off a lot of boxes. I decided to fork the project and try it out, adding bits and bobs of content to see how my writing style would look with the new theme.

I liked it.

Not all was perfect however

As with all things in life, this project would need some work. I found a few things that were lacking which I wanted to fix before releasing a new website. I was determined not to fall into the same trap as before however and structured the work better than I ever had before.

Things I changed

Below you’ll find a list of some of the biggest things I’ve changed.

I added a Dark mode with a little react-toggle to switch between light/dark.
Blog posts now show a header that includes the “back to articles” button and my name.
Added code block theming for both light and dark mode
I added support for tables with a bit of styling around them (slight indent and row styling)
I added the ability to quote others in a beautiful way
Medium like image zooming (click any of the images above)

The posts query bug

Here and there I also encountered some bugs. Most of them got fixed in my copy but one of them was big enough that I felt a need to contribute to the project. The “posts query” bug.

Before the fix, there was a bug in the post-query file that simply looked through all markdown files to find pages with a slug. This, however, is troublesome if you have more than the 3 example pages because the default page sizing will fail. You see, the code that fetches the blogs does so by filtering out real “posts”, but the pagination queries do not. This meant that, given enough pages, the pagination would overflow and tell the user that there were more posts available than there were. If a user were to click on “next” they would be presented with an empty page.

In the end, it was a tiny bug, but I submitted a PR to fix it in the upstream repository.

making it my own

Though the original codebase gave me a great starting point I have decided not to follow the upstream contributions but instead forge my own future. Doing so allows me to keep my codebase clean and allows me to deviate from the choices made by Alexander.

Alexander is slowly introducing more features than I need. One of the newer feature commits (a dark mode) even included his own state management library “diesel” for which I see absolutely no fit in the project.

As such, I have decided to simply skip pulling from upstream and maintain the website myself.

The future is bright

Now that I have this new blog it should be easier for me to write some articles. Let’s hope I will find time to write more than I have in the past 5 years!

My dog, Eddy, died today. He will be missed.

Fri, 30 Sep 2022 00:00:00 GMT

This is Eddy, January 30, 2011

Unfortunately, I’ve had to put my dog Eddy down today. Eddy was my best friend. Not just my best furry friend, but my best friend period. He never failed to bring me joy, he was always there when I needed him and he helped me overcome some of life’s biggest obstacles just by existing.

Putting him down was very hard for me. Eddy had been in a rough shape for at least a week but I wasn’t ready to give up on him. As a matter of fact, I didn’t want to give up until after he was put down. To say I cried like a newborn would be an understatement. I’ve felt pain like this in my life before, and I swore I would never let it happen again but Eddy managed to nestle his way too deep in my heart.

I love you Eddy. Always have. Always will. Rest in peace my friend.

Eddy

2009-10-18 - 2022-09-30

Why Eddy had to be put down

Such a cute boy with his ribbon, July 2016

Of course this isn’t the first time Eddy has had some issues with his health. During the course of a life many things can happen and some of those will inevitably be bad. I’ve included a picture of the time Eddy had to carry a ribbon through his torso to prevent the wound from closing. Not the prettiest sight but it was worth it because he pulled through. Apart from that Eddy also had to take medicine twice a day for various other illnesses, and he was very good at hiding his pills too… Later Eddy lost a lot of his teeth and also a big part of his vision.

The issues with Eddy were much more serious this time though. Eddy has had stomach ulcers for a long time now and whilst they are somewhat treatable with medication they will almost always develop to become a bigger issue. In his last few days Eddy began spewing blood, having very dark excrements and generally being a tad cranky. Not traits usually associated with the happy little dog I knew. Just before Eddy was put down the veterinarian pointed out a hard lump in his stomach, thought to be a cancerous growth. I felt it and remembered feeling something very similar before. I knew then that we were making the right call.

mixed feelings

I am not often an emotional wreck, but today I was. Though, after some thought I realized something that I’d realized before:

For something to cause me so much pain it had to have brought me joy. Memories are fading in and out, my vision goes blurry. I’m coming short of breath… The only thing keeping me sane is knowing I have memories of us, the pictures captured a mere glimpse of it. But my heart captured everything.

- from the poem thoughts by Rick van Lieshout

Very professional doggy. Early 2022

And though those feelings are true. I also can’t help but shake the sorrow I feel. I realize, all too well, that I’ve lost the last thing on this planet that I chose to truly love. Family ties are great and all, but I chose to spend my life with Eddy. I chose to take care of him and to love him until I couldn’t anymore.

A little over 2 years ago I moved out of my parent’s house and I unfortunately couldn’t take Eddy with me. Not only because my job doesn’t really allow me to take the best emotional care (9-5..) of a dog but also because Eddy is old. Eddy simply didn’t want to move. He knew his surroundings, even though he didn’t see very well, and didn’t really want to change them up. Though I went to see him every week (and my parents too…) I still didn’t like being away from him for the rest of the week.

I have debated getting a pet of my own. But as I said before I would feel sorry for the animal because I wouldn’t be home much. I also thought of getting a cat, but I just like dogs more. Now that Eddy has left me completely I will just have to see where life takes me. I sure could use a buddy at times so maybe I will end up adopting a pet anyway.

Farewell

In the end, though, I don’t think anything can truly replace Eddy. I got Eddy an urn so I can have him in my life for the rest of mine. Sleep well, friend.

Eddy peacefully "sleeping" post mortem

The urn

It took way too long for the urn to be filled with pictures but in the end, I’ve gotten it done and I hung Eddy’s urn in my bedroom.

A lovely memory of Eddy

Server management tips: Keep 1 sata port available

Fri, 01 Dec 2017 00:00:00 GMT

Servers… Both a joy and a pain. This week my server has just been a pain though… What should’ve been a relaxing night with some easy peasy hard drive replacements quickly turned into hours of downtime and lots of yelling and screaming :(

During my holiday stay (at Tropical Islands in Germany, pictures can be found here), I decided I’d replace the last 3 500 GB drives in my server array (don’t ask me why I was thinking about my server on holiday). So, as soon as I got home, I decided to look around and order some of those beautiful WD Red drives. I considered upgrading my parity drive as well, but I’d have to buy 3 TB models and that seemed like more of a hassle than I was willing to take on at the moment. (I should have.)

So, the drives arrived, I ran a quick benchmark followed by a quick drive test and all seemed fine with the first drive. At this point, I decide to open up the server and replace the first 500 GB hard drive. After a lot of gymnastics (trying to read drive labels while the drives are in the drive bays) I finally figured out which drive I should replace and I went to work on that drive. I was careful not to touch any of the other cables to avoid accidentally unplugging them and I thought I had done a good job. I remember thinking: “Drive replaced, cables still connected, not much hassle so far. Let’s turn it on”. And so I did.

Prepare for trouble

My good mood was rudely interrupted by a loud beep followed by three, rapid, beeps. At first, I thought it was the regular old “fan not spinning” error (I use several ways to limit fan speed) but on further inspection, I realised that the fan was spinning just fine. Things were about to turn bad…

The drive I had inserted was fine, the OS picked it up and it was performing well. One of the other drives had failed though… So I shut it back down, reconnected the old drive (the one I replaced) and replaced the failing drive with a brand new one. A parity sync is necessary at this point.. Sigh..

After replacing the “failed” drive and letting the machine run for a while I got an email from my server, emails from my server are usually bad, stating that a drive had failed. At first, I was going to dismiss it as I had just replaced one and figured the email was simply delayed. Upon closer inspection, I found out that another hard drive had failed! So I logged back into the server and inspected the problem. The new drive had failed… even though it previously passed my tests with flying colours!

Something was up… and after countless hours of fighting with my server I finally figured it out. The SATA port on the controller was faulty. It didn’t error, it didn’t smoke, it looked fine. It even held on to the cable just fine (cause of those tiny little lips on SATA cables) so it was kinda hard to find out that the SATA port was faulty.

a random yellow SATA cable

So, with a way to reproducibly “fail” known good drives it was just a matter of plugging the SATA cable into another port and go on with my day… or so I thought…

Little did I know that my server had used up all its SATA ports! I didn’t have a single one left! In the end, I had to go out, pay way too much for a raid card that I didn’t need and work on the server till dawn in order to fix my mistakes.

Please don’t make the same mistake… make sure you have a replacement SATA port (or raid card for that matter).

Live and Learn I guess, Live. And. Learn.

Scala Day 4 - A challenge!

Wed, 05 Apr 2017 00:00:00 GMT

The book doesn’t offer a day 4, my school, however, does. Or rather, it wants us to “create” a day 4. The main objective of day 4 is to “create a smart goal which fits the language and challenges you”. For this purpose I came up with the following goal:

For the “4th” day in the “Seven Languages in Seven Weeks” book, I want to create a sitemap builder which prioritises URLs based on their number of occurrences.

I picked this specific goal because Scala has awesome functionality for collections, as such it should make creating a site map a piece of cake! All the code I created for this fourth day can be found on Github or at the end of this blog post.

My journey with Scala (and the book…)

Scala, for the most part, is pretty ok. The development tools aren’t up to par with those of say Java, Python or even .net but with a bit of hassle, we can make do. The Scala collections are wicked though! I’ve done some really crazy things using the collections already and I’m sure there is more to come. (if anyone ever wants me to write a bit of Scala that is)

The concurrency chapter was a big failure, the book is old (2010) and covers the older API. This meant that I had to cast aside the book and create the examples with the new API myself. A drag, to say the least.

As far as functional goes, my opinion hasn’t changed much. I still prefer OO, maybe because I grew up doing everything the OO way or maybe because functional programming is just not for me. Whatever the case I still prefer OO. But… I’ve gained a new appreciation for the functional languages so I guess you could call this experiment a success.

Anyways, that was it for me guys, hope you enjoyed and see you later!

import java.io.{File, PrintWriter}

import scala.collection.immutable.ListMap
import scala.io.Source

object PageLinkLoader {
  //I stole this regex from somewhere for a previous project, don't remember where

  // regex commented out because pretify can't handle it
  //val hrefRegex = """<a\s+(?:[^>]*?\s+)?href=(["'])(.*?)\1""".r

  // merge two maps
  def mergeMap[A, B](ms: List[Map[A, B]])(f: (B, B) => B): Map[A, B] =
    (Map[A, B]() /: (for (m <- ms; kv <- m) yield kv)) { (a, kv) =>
      a + (if (a.contains(kv._1)) kv._1 -> f(a(kv._1), kv._2) else kv)
    }

  // run through all links on a page and add them to a list
  def getLinks(url: String, urls : Map[String, Int], visited : List[String] = List[String]()) : Map[String, Int] = {

    val content = Source.fromURL(url)(io.Codec("ISO-8859-1")).mkString
    val links = hrefRegex.findAllIn(content).matchData.toList.map(_.group(2).toString)

    // filter out / , # , http://url and anything else not starting with http:// or www.
    val filteredList = links.filter(
        l => l.startsWith("/") ||
        l.startsWith("#") ||
        l.startsWith(url) ||
      l.contains(url.replace(url.split("/").last, ""))
    )

    // create new mutable list and fill with visited
    var newVisited = scala.collection.mutable.ListBuffer.empty[String]
    newVisited ++= visited.filter(p => !p.equals(url))

    //add to newVisited
    filteredList.foreach(s => {
      if(!newVisited.contains(s)){
        newVisited += s
      }
    })

    // convert list to map, count occurrences
    val foldedLinks = filteredList.foldLeft(Map[String, Int]())((map, link : String) =>
      map + (link -> links.count(_.equals(link)))
    )

    newVisited = newVisited.filter(s => !s.equals(url))

    if(newVisited.isEmpty){
      // return merged map
      mergeMap(List(urls, foldedLinks))((v1, v2) => v1 + v2)
    }else{
      getLinks(newVisited(0),mergeMap(List(urls, foldedLinks))((v1, v2) => v1 + v2), newVisited.toList)
    }
  }

}

object MyApp extends App{

  val urls = List(
    "http://servers.rickvanlieshout.com/scalatest"
  )

  def getSiteMap() = {

    var siteMap : Map[String, Int] = Map[String, Int]()

    for (url <- urls) {
      siteMap = PageLinkLoader.getLinks(url, siteMap)
      siteMap.foreach (x => println (x._1 + "-->" + x._2))
      createSiteMap(url, siteMap)
    }
  }

  // creates a sitemap, uses Java which is wicked.
  def createSiteMap(url: String, urls : Map[String, Int]) = {
    // start off with a priority of 1.0 (ranges from 1.0 to 0)
    var currentPriority = 1.0

    //sort descending
    val descUrls = ListMap(urls.toSeq.sortWith(_._2 > _._2):_*)

    // write to a file called "sitemap_for_domain.txt"
    val domainRegex = """^(?:https?:\/\/)?(?:[^@\/\n]+@)?(?:www\.)?([^:\/\n]+)""".r
    val pw = new PrintWriter(new File("sitemap_for_"+domainRegex.findAllIn(url).matchData.toList.map(_.group(1)).mkString+".txt" ))
    //write xml header
    pw.write("\n<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<urlset\n      xmlns=\"http://www.sitemaps.org/schemas/sitemap/0.9\"\n      xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\n      xsi:schemaLocation=\"http://www.sitemaps.org/schemas/sitemap/0.9\n            http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd\">")

    // for each url write an url block with priority, then decrease priority
    descUrls.foreach(s => {
        pw.write(
          "\n<url>\n  " +
            "<loc>" + s._1 + "</loc>\n" +
            "<priority>" + currentPriority + "</priority>" +
            "</url>"
        )
        // decrease priority if possible
        if(currentPriority >= 0.1){
          currentPriority -= 0.1
        }
      }
    )

    // write the closing statement for a sitemap
    pw.write("\n</urlset>")
    pw.close()
  }

  getSiteMap()

}

Scala Day 3 - Concurrency is key!

Tue, 04 Apr 2017 00:00:00 GMT

Day 3 promises to go into “hard things” like XML and concurrency. So far I’ve been liking Scala and if it can really simplify concurrency then it might earn a permanent slot in my toolkit.

XML

We can express XML just as easily as we do a string, the example below shows how to express some XML and how to access its inner text. Note, however, that you need to add the “scala-xml” jar file to your build solution as it no longer comes included with Scala.

scala> val movies =
     | <movies>
     | <movie genre="scifi">Star Trek</movie>
     | <movie genre="fairytale">Star Wars</movie>
     | </movies>
movies: scala.xml.Elem =
<movies>
<movie genre="scifi">Star Trek</movie>
<movie genre="fairytale">Star Wars</movie>
</movies>

scala> movies.text
res0: String =
"
Star Trek
Star Wars
"

We can use the ” \ ” operator to search for specific entries, which we can then access just as with a list:

scala> val movieNodes = movies \ "movie"
movieNodes: scala.xml.NodeSeq = NodeSeq(<movie genre="scifi">Star Trek</movie>, <movie genre="fairytale">Star Wars</movie>)

scala> movieNodes(0)
res1: scala.xml.Node = <movie genre="scifi">Star Trek</movie>

To check the “genre” attribute of the first item in movieNodes we run:

scala> movieNodes(1)
res3: scala.xml.Node = <movie genre="fairytale">Star Wars</movie>

scala> movieNodes(1) \ "@genre"
res4: scala.xml.NodeSeq = fairytale

As we can see it returned the correct genre, fairytale, for the movie “Star Wars”. So far Scala delivers on its promise to make “hard things” like XML easy. Let’s see what else we can do.

Pattern matching, guards and Regex

Scala will use pattern matching often, such as when you parse XML or pass messages between threads. The simplest form of pattern matching can be achieved with:

def doChore(chore: String): String = chore match {
  case "clean dishes" => "scrub, dry"
  case "cook dinner" => "chop, sizzle"
  case _ => "whine, complain"
}
println(doChore("clean dishes" ))
println(doChore("mow lawn" ))

We define two chores and an alternative. We then call the method with a valid and invalid parameter. This will match the valid one and return “scrub, dry”, the second won’t match and will return “whine, complain”.

Scala also supports guards in the case statement. Using this to calculate factorials, for instance, we can filter out weird numbers (like 0 and negatives) while matching all the other positive numbers to x, and thus return it’s factorial.

def factorial(n: Int): Int = n match {
  case 0 => 1
  case y if n < 0 => 0
  case x if x > 0 => factorial(n - 1) * n
}
  println(factorial(-2))
  println(factorial(0))
  println(factorial(5))

We can also use regular expressions to do our matching:

scala> val reg = """^(F|f)\w*""".r
reg: scala.util.matching.Regex = ^(F|f)\w*

scala> println(reg.findFirstIn("Fantastic"))
Some(Fantastic)

scala> println(reg.findFirstIn("not Fantastic"))
None

We can even combine XML and the matching features like so:

(movies \ "_" ).foreach { movie =>
  movie match {
    case <movie>{movieName}</movie> =>
      println(movieName)
    case <short>{shortName}</short> =>
      println(shortName + " (short)" )
  }
}

val movies = <movies>
   <movie>The Incredibles</movie>
   <movie>WALL E</movie>
   <short>Jack Jack Attack</short>
   <short>Geri's Game</short>
</movies>

This will go through every item in the tree (_) and match it to either movie or short. If it’s a movie it will just print its text, otherwise, it’ll print its text + (short).

In order to get this to work however I had to download the scala.xml library, as it no longer comes included with Scala since Scala 2.11.

Concurrency

Finally! We get to do something with concurrency, now Scala can really show me what it’s worth!

The book starts off by telling me about Actors, which have pools of threads and queues, and message passing. When you send a message (using the ! operator) you place an object on its queue. The actor then reads the message and takes action. Usually, the actor uses a pattern matcher to detect what it has to do before it starts doing something.

The book provides a sample application but that plain doesn’t work. Upon investigating the issue I discovered that Scala’s built-in concurrency feature is deprecated in favour of Akka. This saddens me beyond belief. I have already worked with Akka and had expected Scala to offer me something else. This also means that the rest of the book is not going to be useful anymore.

Stubborn as I am, even with a closing deadline, I decided I wasn’t going to let a stupid book stop me and decided to update the code to work with Akka. All of which can be found at Github.

This disaster can be summed up by a fantastic quote though (and I love quotes):

All that is necessary is to accept the impossible, do without the indispensable, and bear the intolerable.

- Norris, Kathleen, As quoted in: The Litchfield Co-operator, Issues 116-163 (1945)

Self-study

The theoretical question for the day is: For the sizer program, what would happen if you did not create a new actor for each link you wanted to follow? What would happen to the performance of the application?

And the answer is: it would be just as slow as the sequential one.

The practical assignment is: Take the sizer application and add a message to count the number of links on the page.

I did this by creating another Loader and calling that in my UrlActor. The code can, once again, be found on Github or in the code block below.

object PageLinkLoader {

  def getLinks(url: String) : Int = {
    val content = Source.fromURL(url)(io.Codec("ISO-8859-1")).mkString
    val links = hrefRegex.findAllIn(content).matchData.toList.map(_.group(2))

    links.size
  }

  //I stole this regex from somewhere for a previous project, don't remember where
  val hrefRegex = """<a\s+(?:[^>]*?\s+)?href=(["'])(.*?)\1""".r

}

So that’s it, right? Wrong! My school assignment requires me to do one more assignment. What assignment is that? Well you’ll just have to wait on the next blog to find out! Laters!

Scala Day 2 - Let's get functional

Mon, 03 Apr 2017 00:00:00 GMT

I feel like I covered a lot in the first chapter but that I haven’t yet found the real power of functional programming. Day 2 promises to change that by diving right into functional programming.

Val vs Var

I’ve already looked at this on day 1, looking back at the chapter I realize I wasn’t yet meant to find out about the difference between “val vs var”. In short, val is immutable while var is mutable.

The remainder of the chapter emphasizes the importance of using val, especially while designing an application with concurrency in mind.

This basic design philosophy is the key element that differentiates functional programming from object-oriented programming: mutable state limits concurrency.

- Tate, Bruce, Seven Languages in seven weeks (p 155)

Collections

In the first blog I already pointed out some of the features I had discovered about collections. I used some of these awesome features in my day 1 self-study.

The second chapter goes into more detail about collections. First up are lists.

Lists

Scala’s lists, of type List, are ordered collections of elements with random access. One of the things the book immediately goes into is the ability to store different types of objects in the list. Take a look at the interpreters response when I try to create a list of two strings and an integer:

scala> List("one", "two", 3)
res6: List[Any] = List(one, two, 3)

This returns a list with data type “Any”, which is the catchall data type for Scala. To access an item we use the ”()” operator. Think of it as the list being a method on which you “call” a get function. Doing so will return an “Any” object, which to me is weird because it’s type could be inferred as well.

scala> List("one", "two", 3)(2)
res7: Any = 3

Trying to access a number outside the list will throw either a “NoSuchElement” exception (if the index specified is too high) or a regular old “IndexOutOfBoundsException”. (if the index is below 0)

Sets

A set is like a list, but it doesn’t have any explicit order. We can specify a set with the Set keyword like so:

scala> val animals = Set("lions", "tigers", "bears")
animals: scala.collection.immutable.Set[java.lang.String] = Set(lions, tigers, bears)

Adding or subtracting from it is as easy as using the + or - operator:

scala> animals + "armadillos"
res0: scala.collection.immutable.Set[String] = Set(lions, tigers, bears, armadillos)

scala> res0 - "tigers"
res2: scala.collection.immutable.Set[String] = Set(lions, bears, armadillos).

Combining sets can be done with the ”++” operator (or list.union) and the ”—” operator (or list.diff) will return a list of differences between the two lists:

scala> animals ++ Set("armadillos", "raccoons")
res3: scala.collection.immutable.Set[String] = Set(bears, tigers, lions, armadillos, raccoons)

scala> animals -- Set("lions", "bears")
res4: scala.collection.immutable.Set[String] = Set(tigers)

Set intersection can be done with “List.intersect” like so:

animals.intersect(Set("armadillos","raccoons","lions","tigers"))
res8: scala.collection.immutable.Set[String] = Set(lions, tigers)

As I’ve mentioned before, Sets, unlike lists, are independent of order. This rule will mean that equality for sets is different. Therefor evaluating two sets with the same elements in a different order will return true:

scala> Set(1, 2, 3) == Set(3, 2, 1)
res9: Boolean = true

Maps

A map is a key-value pair, similar to hashmaps in Java. Specifying a map of ordinal numbers can be done with:

val ordinals = Map(0 -> "zero", 1 -> "one", 2 -> "two")
ordinals: scala.collection.immutable.Map[Int,String] = Map(0 -> zero, 1 -> one, 2 -> two)

We can also specify the type for a HashMap like so:

import scala.collection.mutable.HashMap
val map = new HashMap[Int, String]

We can then add items like so:

map += 4 -> "four"
map += 8 -> "eight"

List methods

Scala supports anonymous functions, I’ve used this to solve some problems in the Tic-Tac-Toe game from the first blog.

Apart from the “foreach” method, there are some other useful list methods:

Function	What it does
List.head	Returns the first element
List.tail	Returns all but the first element
List.last	Returns the last element
list.init	Returns all but the last element (recursion)
List.reverse	Reverses the list
List.drop(2)	Removes the first two objects.

Count, map filter and Others

Scala has many other functions that manipulate lists in various ways. I won’t go over these but the code examples below should give you an idea of the possibilities.

scala> val words = List("peg", "al", "bud", "kelly")
words: List[String] = List(peg, al, bud, kelly)

scala> words.count(word => word.size > 2)
res0: Int = 3

scala> words.filter(word => word.size > 2)
res1: List[String] = List(peg, bud, kelly)

scala> words.map(word => word.size)
res2: List[Int] = List(3, 2, 3, 5)

scala> words.forall(word => word.size > 1)
res3: Boolean = true

scala> words.exists(word => word.size > 5)
res4: Boolean = false

We can also parameterise our anonymous functions so that we can sort a list based on the first letter of each word. We do that like this:

words.sortWith((s, t) => s.charAt(0).toLower < t.charAt(0).toLower)

We can also sort by word size:

words.sort((s, t) => s.size undefined

foldLeft

The foldLeft method takes an initial value and a code block. It will take the initial value and for each item in the list run the code block passing back in the calculated value. Let me show you:

val list = List(1, 2, 3)
val sum = (0 /: list) {(sum, i) => sum + i}

Initially, /: takes the initial value, 0 , and the first element of list , 1 ,and passes them into the code block. sum is 0 , i is 1 , and the result of 0 + 1 is 1 .
Next, /: takes 1 , the result returned from the code block, and folds it back into the calculation as sum . So, sum is 1 ; i is the next element of list , or 2 ; and the result of the code block is 3 .
Finally, /: takes 3, the result returned from the code block, and folds it back into the calculation as sum . So, sum is 3 ; i is the next element of list , or 3 ; and sum + i is 6 .

We can achieve the same thing by using the foldLeft method like so:

list.foldLeft(0)((sum, value) => sum + value)

Self-Study

I’m starting to like Scala more and more, the practical challenge for this week was really fun albeit a bit simple. The theoretical questions for day 2 are:

A discussion on how to use Scala files
What makes a closure different from a code block

Once again these questions are trivial if you’ve actually paid attention while reading the book. Anyway, here are the answers:

The results of the second day’s practical assignments can found on Github or below:

/**
   Self-Study day 2, practical assignments:

   • Use foldLeft to compute the total size of a list of strings.

   • Write a Censor trait with a method that will replace the curse words
     Shoot and Darn with Pucky and Beans alternatives. Use a map to
     store the curse words and their alternatives.

   • Load the curse words and alternatives from a file.

*/

trait Censor {

  val curseWords = Map("shoot" -> "pucky", "darn" -> "beans")

  /**
    * Awesome function which will build a curseWords map from a file.
    * @param input string input
    * @param path curseWords file
    * @return clean string
    */
  def censorTextUsingFile(input: String, path: String) : String = {
    // neat little statement which will read a file from a path, get its lines
    // for each line it will split on "=" and put the values in the map
    val myMap = io.Source.fromResource(path).getLines.foldLeft(Map[String, String]())((map, line) =>
      map + (line.split("=")(0) -> line.split("=")(1))
    )

    censorText(input, myMap)
  }

  /**
    * censors input text using the builtin curseWords map or @param map
    * @param input string to be censored
    * @param map optional, map of curse words
    * @return censored string
    */
  def censorText(input: String, map: Map[String, String] = curseWords) : String ={
    val words = input.split(" ")

    words.map(word => map.getOrElse(word.toLowerCase, word)).mkString(" ")

  }
}

object MyApp extends App with Censor{

  // print combined size of a list of strings
  val list = List("one", "two" , "three")
  println(list.foldLeft(0)((sum, value) => sum + value.length))

  val strToCensor = "Shoot !, this darn thing won't work!"

  // censor using builtin map
  println(this.censorText(strToCensor))

  // censor using curseWords.txt file from resources folder
  println(this.censorTextUsingFile(strToCensor, "curseWords.txt"))

}

Scala Day 1 - The Basics

Sun, 02 Apr 2017 00:00:00 GMT

So day 1 has arrived, I’ve opened the book and I’ve started looking at the assignments in front of me. The book starts off by telling us how to install Scala so that is what I’ll start with today.

Installing Scala

Being on Linux the first thing I did was use my package manager of choice (Pacman) to check whether there was a Scala package in my repositories. Sure enough, it found a package (20MB in size) and proceeded to download and install said package.

While Pacman started collecting all the little dots, I took a quick look at the Scala download page. On there I am greeted with a friendly message telling me the binary for my system is ready to download. Right below the download button is a link on how to install Scala. Curiosity got to me and I decided to open the link. The installation steps briefly explain that there are 2 binaries in the compressed folder which are of importance: “scala” and “scalac”. To anyone coming from Java, these should be fairly familiar seeing as Java uses “java” and “javac” for their interpreter and compiler respectively. (note: the installation steps assume you know how to add PATH variables).

The next message on this page tells me about a popular build tool called sbt so I went ahead and installed that as well.

The final step on this page recommends me to install ”The Scala IDE” or use the IntelliJ plugin. Seeing as I adore the JetBrains product line I’ve opted to choose the latter.

While browsing the downloads page I noticed Pacman had beat his level. (a.k.a collected all the dots. a.k.a the install has finished) Which means it’s high time for me to continue on with the assignments.

Scala types

The next step in the book guides me through using the interpreter to execute basic bits of code. These range from a simple hello world to entering some arithmetics. In the code block below you can see both my input and the interpreter’s output.

scala> println("hello, surreal world")
hello, surreal world

scala> 1 + 1
res1: Int = 2

scala> 5 + 4 * 3
res2: Int = 17

scala> 5. + (4.*(3))
res3: Int = 17

Two things struck me as odd about these results. First up the word “Int” is written with a capital I. This is weird to me because in Java “int” is a base type, not a class (it uses the Integer wrapper class). Next up is the “ResX” line, does that just mean result or is there something more to find out about it? After a quick Google search I found out the “res” is an immutable variable which we can use like so:

res0: Int = 10

scala> res0 + 10
res1: Int = 20

Type inference, arrow notation, “var vs val” and GitHub

At this point the code is getting slightly more complicated and typing it all directly into the interpreter would be a fool’s effort. Therefore I’ve switched to using IntelliJ with the scala plugin and .scala files. All the files can be found on GitHub.

After fiddling a bit with the examples I’ve found a few things I already like in Scala:

Type inference. To me, not having to specify the type of a variable is strangely satisfying.
Anonymous function syntax support (a.k.a arrow notation).
First class ranges
1 line methods without braces
companion objects (extending.scala on Github)
No semicolons! (although I sometimes use them by accident)

Inheritance is weird

Inheritance is one of the many joys from the Object-Oriented programming paradigm and I’m glad to read Scala supports inheritance too. The way Scala does it, however, is rather weird in my opinion. Take a look at the following code:

class Person(val name: String) {
  def talk(message: String) = println(name + " says " + message)
  def id(): String = name
}
class Employee(override val name: String, val number: String) extends Person(name) {

  override def talk(message: String) {
    println(name + " with number " + number + " says: " + message)
  }

  override def id():String = number.toString

}

As you can see we have to specify the parameter list of the class we want to extend. Also, note that we don’t have to specify the type of said parameters. Another peculiarity is the “override” keyword. Apparently, it is mandatory to prevent the creation of new methods due to mis-spelling … lame.

Traits

Traits, in short, resemble a Java interface combined with an implementation.

We will start by defining a trait “Bad” for the Person class we’ve defined earlier:

trait Bad{
  def curse() = println("Object oriented languages are cool too!");
}

Next, we’ll override the person class with a new class and add the “Bad” trait:

class BadPerson(override val name:String) extends Person(name) with Bad

Now if we create a new object of BadPerson we can call the “curse” method.

val badPerson = new BadPerson("Mr. bad")
badPerson.curse

We can also add multiple traits to a class by repeating “with TRAIT” like so:

class IndecisivePerson(override val name: String) extends Person(name) with Nice with Bad

Self-Study

At the end of every day, the book ends with a few assignments, some are theoretical and some are practical.

Day 1’s theoretical questions are:

Find the Scala API
Find a comparison of Java and Scala
Find a discussion of “val” vs “var”

My answers to these are:

Finally, we get to move on to the practical assignment for the week. For the assignment, I have to create a “Tic-tac-toe” game for 2 players.

Writing this game has been really interesting. I tried using as few variables as possible and I tried to use Scala-specific features. One of these features I have already fallen in love with: collections. Scala collections have methods similar to LINQ in c# and they are extremely useful. Examples of these collection methods can be found in the “boardHasWinner” method in the code below.

The complete code for this game, including the bonus problem, can also be found at Github.

object TicTacToe {

  var currentPlayer = 'X';
  val playerOne = 'X'
  val playerTwo = 'O'


  val board = Array(
    Array('_', '_', '_'),
    Array('_', '_', '_'),
    Array('_', '_', '_')
  )

  /**
    * Flips current player to the other player.
    */
  def flipPlayers = {
    if(currentPlayer.equals(playerOne)){
      currentPlayer = playerTwo
    }else{
      currentPlayer = playerOne
    }
  }

  /**
    * read user input and play a move
    * @param error set to true if user made a mistake and has to re-enter his position
    */
  def readLine(error : Boolean = false) : Unit = {
    if(error){
      System.err.println("Please enter a value ON the grid which has no mark yet.")
    }

    println("Please enter the x position you want to place your mark at")
    val x = scala.io.StdIn.readLine().toInt

    println("Please enter the y position you want to place your mark at")
    val y = scala.io.StdIn.readLine().toInt

    if((x > 0 && x < 4) && (y > 0 && y < 4) ){
      play(x,y)
    }else{
      readLine(true)
    }
  }

  /**
    * checks whether pos(x,y) is a valid position
    * if it's valid it will put currentPlayer on that position
    * if it isn't it will ask the user for new input using the readLine method
    * @param x coordinate
    * @param y coordinate
    */
  def play(x : Int, y : Int) = {
    if(board(x - 1)(y - 1).equals('_')){
      board(x - 1)(y - 1) = currentPlayer
      flipPlayers
    }else{
     readLine(true)
    }
  }

  /**
    * Checks whether the board is full
    * @return true if board is full, else false
    */
  def boardIsFull : Boolean = {
    board.foreach { row =>
        // if any row contains an underscore we're obv not full yet so we can return false;
        if(row contains '_'){
          return false;
        }

    }
    return true;
  }

  /**
    * main game loop
    */
  def run = {
    while(!boardIsFull && !boardHasWinner){
      printBoard
      readLine()
      if(boardHasWinner){
        flipPlayers // reverse last player flip
        println("Congrats " + currentPlayer + " you have won this game of Tic-Tac-Toe")
      }
    }
    if(!boardHasWinner){
      println("Sorry, it's a draw!")
    }
    // print the board once more so the players can see the final score
    printBoard
    // exit execution
    System.exit(0)
  }

  /**
    * Checks whether the board has a winner
    * @return true if board has a winner else false.
    */
  def boardHasWinner : Boolean = {

    //check horizontals
    board.foreach {row =>
      if(row(0) != '_' && row.forall(c => c.equals(row(0)))){
        return true
      }
    }

    // check diagonals
    val topLeftBottomRight = Array(board(0)(0), board(1)(1), board(2)(2))
    val bottomLeftTopRight = Array(board(0)(2), board(1)(1), board(2)(0))

    if(topLeftBottomRight(0) != '_' && topLeftBottomRight.forall(c => c.equals(topLeftBottomRight(0)))){
      return true
    }
    if(bottomLeftTopRight(0) != '_' && bottomLeftTopRight.forall(c => c.equals(bottomLeftTopRight(0)))){
      return true
    }
    // if no winner has been found, return false.
    return false
  }

  /**
    * Prints a visual representation of the board and who's turn it is
    */
  def printBoard = {
    board.foreach { row => println("" + row(0) + " | "
      + row(1) + " | "
      + row(2) )}

    println(currentPlayer + " is playing")
  }

}

// let's play!
TicTacToe.run

Scala Day 0 - What is Scala?

Thu, 30 Mar 2017 00:00:00 GMT

One of the final assignments for a school course called “APP” (Algorithms Programming language & Paradigms) was/is to learn a new programming language using the book ”Seven languages in Seven weeks“.

The idea of this book is that you can get comfortable with a programming language in just a week. As you can probably guess from the title the book covers 7 languages in total and I will be covering at least 1 (Scala).

What is Scala

Scala is a hybrid language that tries to bridge the gap between object-oriented languages like Java and functional languages like Haskell.

Why did I pick Scala for the assignment?

I have never really liked functional languages (except Javascript) much. It’s a hard paradigm for me to wrap my head around in most languages and it gets tiring quickly. I thought having an assignment (for which I will be graded) to focus on might help me stay on the right path.

So… if you wanted to go functional why not go Haskell you ask? Well, purely functional programming is something which just doesn’t “speak” to me, combining it with an OO approach might just be the thing that makes the gears click. At the very least it should provide a gradual learning curve to apply functional programming ideas to my code.

Besides my personal reasons, there are some objective reasons why Scala is “better” than Haskell:

Scala is much more popular than Haskell, as such chances of encountering it “in the wild” are severely increased.
Scala painlessly integrates with pre-existing java libraries.
Scala uses the Java Virtual Machine (JVM), this has numerous benefits.
Like Java, Scala is statically typed and the syntax somewhat resembles that of Java too.

Well that’s it for today folks, check back in for the next blog within a few days!

My extended leave, status updates and other stuff

Fri, 02 Sep 2016 00:00:00 GMT

So….. last blog post was in December of last year. Does that mean the blog is dead? Well I’m happy to report that is NOT what it means! In this blog post I’ll address why I’ve been on an extended leave and I have some exciting news to boot!

Good news always comes first!

Before I start on my rant about being away for so long I’d better tell you all some exciting news. And boy do I have quite a bit of it!

First of all, if you hadn’t noticed yet, the personal website has gotten a facelift! I have added a new “Software” page, this page will contain software I made without a contract and open source projects I’ve worked on. Furthermore, I have updated the CV with some new job information and a couple of snazzy logos. The Portfolio page got trimmed down a bit (moved some stuff to Software) and the contact page has gotten a nice new look with an actual form instead of just social buttons. Lastly, I added an “Other sites” drop-down menu. This menu will house links to various other websites of which I am the owner / co-owner. As some of those keen-eyed people among you will have noticed I also upgraded my “Disclaimer” page and added a new “RSS feed” to the footer of the site.

All in all, quite a few changes which will allow me to further separate Mi-soft.nl from my personal website. (update to Mi-soft.nl expected somewhere later this year hehe)

More good news!?

Oh yes! I have recently started working on Three, yes Three, new projects. I will probably blog about all of these projects but in case I don’t here’s a little overview:

First up is a new Python (Flask) REST API which makes getting Linux OS information a breeze! I started this project because I wanted to recreate my old hardware info system (written in Java) and generally improve it overall. I started out working on this by myself but soon joined up with my old friend Janco (more on him later). We haven’t done all too much about it but in its current state it can already serve most, if not all, CPU and Memory information. Though the best feature of this new API is arguably its name. I named it after the person I have cherished for over six years now, Sammy.

I also finally bit the bullet regarding “Linfinity”. For a while now I’ve been wanting to create a “software install script generator” for Linux which I can use to install Linux software easily on a multitude of systems. Linfinity will be a web tool which will be able to generate software install scripts for various different distributions (aiming for at least: Arch, Ubuntu and Raspbian).

Lastly, I have some “maker” news. A friend and I have decided to try our hand at creating a Raspberry Pi workplace for the masses. We found an absolutely perfect display to use in combination with the Raspberry pi and have devised a way to mount it and the Rasberry Pi in a single chassis. Cables will be mostly routed within the unit and it all can be powered by a single standard 2.1mm adapter! More news about this soon!

Moving away from Digitalocean

As of today I no longer have any Digitalocean droplets. Not because they offer a bad service but because they are too expensive for what I was using them for. The only real purpose my droplet still had was handling go8.nl and it’s traffic. I have since moved go8.nl over to Sohosted purely because it is a lot cheaper :).

The rekindled friendship?

As some of you might know Janco and I completely split up about a year and a half ago after a rather lengthy fight between us. In this year and a half apart we both calmed down a bit and after a good talk we’ve decided to pick things up again. So far everything is going well and we are working on keeping it that way. Even though programming together will certainly add to both of our stress levels , and probably lead to a couple of smaller fights, we have decided to team up to create Sammy, my new REST API for extracting Linux OS information.

One of my other good friends (Jelle) also happened to come by some extra free time these past months and we’ve been using that time to explore the magical world of Tamriel (ESO online) and kick some butt (Awesomenauts) together.

So all these things combined have been taking up most of my free time and motivation. Which leaves none for writing blogs unfortunately. I will try to improve upon this though.

Well that’s it for me folks, Have a great day and may happiness surround you :)

TekTree part 4 - programming

Thu, 24 Dec 2015 23:00:00 GMT

In the last installment of the series I show you how to program the arduino to work with the TekTree we built. I would advise each and every one of you to download my sketch and try it but after that start experimenting with your own designs!

Testing

Before we start writing complicated code we first need to design a very simple test sketch for the leds. The script I used can be found below or at Github under the following directory: Testing -> LedTest

The code is quite simple. It sets all led’s as output and runs through them. back and forth

 /*
   Rick van Lieshout
   24-12-2015
   LICENSE: MIT
 */

 const int DELAYTIME = 100;
 const int NUMBEROFPINS = 11;
 const int pinArray[NUMBEROFPINS] = {0,1,2,3,4,5,6,7,8,9,10};

 void setup() {
   //initiate pins
   for (int thisPin = 0; thisPin < NUMBEROFPINS; thisPin++) {
     pinMode(pinArray[thisPin], OUTPUT);
   }
 }

 void loop() {
   // loop from the lowest pin to the highest:
   for (int thisPin = 0; thisPin < NUMBEROFPINS; thisPin++) {
    // turn the led on:
    digitalWrite(pinArray[thisPin], HIGH);
    // wait for a bit
    delay(DELAYTIME);
    // turn the led off:
    digitalWrite(pinArray[thisPin], LOW);
   }
   // loop from the highest pin to the lowest:
   for (int thisPin = NUMBEROFPINS - 1; thisPin >= 0; thisPin--) {
    // turn the led on:
    digitalWrite(pinArray[thisPin], HIGH);
    // wait for a bit
    delay(DELAYTIME);
    // turn the led off:
    digitalWrite(pinArray[thisPin], LOW);
   }
 }

Assuming all went well you should now see the led’s come on 1 by 1 and create a little wave effect (first pattern down!)

Now that we know it all works let’s clean up the code and add some pattern handling.

The finished code / result

The complete code can once again be found on github or directly below the video. That’s it for now. Merry christmas and a happy new year folks!

The code that makes it work

/*
  Rick van Lieshout
  24-12-2015
  LICENSE: MIT
*/

/** Constant value's, these have to be changed manually  **/
const int MAXSEQUENCES= 20;     // number of steps in the largest sequence. Could make it 999 but that would affect performance
const int NUMBEROFPINS = 11;    // number of leds
const int NUMBEROFPATTERNS = 5; // number of patterns specified
const int DELAYPOT = 0;         // number of pin that pot is connected to
const int BUTTONPIN = 12;       // number of pin that button is connected to
const int pinArray[NUMBEROFPINS] = {0,1,2,3,4,5,6,7,8,9,10}; // pins where leds are connected

/** Regular variables, program can alter these **/
int previousbuttonstate = 0;  // used to detect whether button is pressed
int buttonstate = 0;          // used to detect whether button is pressed
int pattern = 0;              // pattern to start with

/** massive array which holds the patterns **/
const byte LEDsequence [NUMBEROFPATTERNS][MAXSEQUENCES][NUMBEROFPINS] = {
  {
  {0,0,0,0,0,0,0,0,0,0,0},
  },
  {
  {1,1,1,1,1,1,1,1,1,1,1},
  },
  {
  {1,0,0,0,0,0,0,0,0,0,1},
  {0,1,0,0,0,0,0,0,0,0,1},
  {0,0,1,0,0,0,0,0,0,0,1},
  {0,0,0,1,0,0,0,0,0,0,1},
  {0,0,0,0,1,0,0,0,0,0,1},
  {0,0,0,0,0,1,0,0,0,0,1},
  {0,0,0,0,0,0,1,0,0,0,1},
  {0,0,0,0,0,0,0,1,0,0,1},
  {0,0,0,0,0,0,0,0,1,0,1},
  {0,0,0,0,0,0,0,0,0,1,1}
  },
  {
  {1,0,0,0,0,0,0,0,0,0,1},
  {0,1,0,0,0,0,0,0,0,0,1},
  {0,0,1,0,0,0,0,0,0,0,1},
  {0,0,0,1,0,0,0,0,0,0,1},
  {0,0,0,0,1,0,0,0,0,0,1},
  {0,0,0,0,0,1,0,0,0,0,1},
  {0,0,0,0,0,0,1,0,0,0,1},
  {0,0,0,0,0,0,0,1,0,0,1},
  {0,0,0,0,0,0,0,0,1,0,1},
  {0,0,0,0,0,0,0,0,0,1,1},
  {0,0,0,0,0,0,0,0,1,0,1},
  {0,0,0,0,0,0,0,1,0,0,1},
  {0,0,0,0,0,0,1,0,0,0,1},
  {0,0,0,0,0,1,0,0,0,0,1},
  {0,0,0,0,1,0,0,0,0,0,1},
  {0,0,0,1,0,0,0,0,0,0,1},
  {0,0,1,0,0,0,0,0,0,0,1},
  {0,1,0,0,0,0,0,0,0,0,1}
  },
  {
  {1,0,0,0,0,0,0,0,0,1,1},
  {0,1,0,0,0,0,0,0,1,0,1},
  {0,0,1,0,0,0,0,1,0,0,1},
  {0,0,0,1,0,0,1,0,0,0,1},
  {0,0,0,0,1,1,0,0,0,0,1},
  {0,0,0,1,0,0,1,0,0,0,1},
  {0,0,1,0,0,0,0,1,0,0,1},
  {0,1,0,0,0,0,0,0,1,0,1},
  {1,0,0,0,0,0,0,0,0,1,1}
  }
};

/** Setup only gets run once **/
void setup(){
  //set all pins as input/output
  initializePins();
}

/** Loop is constantly running **/
void loop() {
  // read button state
  buttonstate = digitalRead(BUTTONPIN);
  // if button state changed
  if(buttonstate != previousbuttonstate){
    previousbuttonstate = buttonstate;
     // check value and add to pattern
     if(buttonstate == HIGH){
      if(pattern +1 == NUMBEROFPATTERNS){
        pattern = 0;
      }else{
        pattern++;
      }
     }
  }

  // Loop through all the arrays and toggle pins on or off.
  for (int sequence=0;sequence<getSequences(pattern);sequence++) {
    for (int pin=0;pin<NUMBEROFPINS;pin++) {
      digitalWrite(pinArray[pin], LEDsequence[pattern][sequence][pin]);
    }
    // delay between sequences, should use millis if you want instant response on the button
    delay(getDelay(DELAYPOT));
  }
}

/** Count the number of sequences in a specific pattern **/
int getSequences(int pattern){
  byte amount = 0;
  for(byte i = 0; i<MAXSEQUENCES; i++){
    int total = 0;
    for(int pin=0; pin<NUMBEROFPINS; pin++){
      total+=LEDsequence[pattern][i][pin];
    }
    if(total>0){
      amount++;
    }
  }
  return amount;
}

/** get value of pot **/
int getDelay(int potnr){
  return analogRead(potnr);
}

/** Set all pins as either output or input **/
void initializePins(){
  pinMode(BUTTONPIN, INPUT);
  for (int pin=0;pin<NUMBEROFPINS;pin++) {
    pinMode(pinArray[pin], OUTPUT);     // we make all the declarations at once
  }
}

TekTree part 3 - Wiring

Thu, 24 Dec 2015 00:00:00 GMT

In the 3rd part of my TekTree project I’ll show you guys how to wire everything. This time I even used separate colours for the power, ground and signal wires. So no more complaining! ^_^ All in all I think the result looks absolutely wicked. Even without the Arduino case which has yet to come in the mail. (and probably won’t fit)

Wiring part 1 - schematics

Before I started to jam wires in every nook and cranny I could find I had to come up with a plan. Most of that plan has been put into a “Fritzing” sketch which can be found below and on Github. The only thing I couldn’t figure out how to model on fritzing is the actual Christmas tree and the power cord for the Arduino.

The Arduino will get power from a regular adaptor. The only modification I made to the adapter was putting a button on one of the wires. The only reason I did that is because I’m too lazy to unplug the TekTree when I want to turn it off :)

Why so many wires?

I want the project to be “manageable” for everyone. To do this I kept the wiring as simple as possible. Every LED has 2 prongs, 1 goes to ground 1 goes to the Arduino. Simple. Same goes for the potentiometers, 3 legs, 1 power 1 ground 1 signal. The only leds I did bundle together (in terms of wiring) is the star. I did this to create a single star instead of multiple led’s.

Well you could’ve made it way prettier… and

Well yes, as all of us know I’m perfectly capable of designing a refined build. I didn’t want to go with a pcb this time however because that would make the barrier to entry way higher on this project. And that is something I definitely don’t want.

I also didn’t “group” all my ground wires together. (led -> led -> led -> board) This would’ve made the project a lot neater but it wouldn’t be anywhere near as service friendly. (which was kinda the idea…)

Well then, I feel like I prepared you guys enough to show the (complicated looking, but very simple) wiring. Check it out below!

Wiring part 2 - Actual wiring

Below you’ll find the pictures I took while I was wiring the TekTree.

TekTree part 2 - Lasercutting

Sat, 19 Dec 2015 23:00:00 GMT

Last friday I actually got to cut the parts for my TekTree! So excited! This is by far the biggest “piece” I have ever laser cut but luckily everything went well. This in part is thanks to my brilliant friend Ron who helped me create the sketches for the laser cutter. Thanks again Ron!

Top tip! You don’t have to laser cut the designs. Just print the designs with a regular printer and stick it on some wood. Then just use something like a bandsaw to cut the pieces!

Laser cutting video

I’ve gotten so much positive feedback on my last laser cutting video that I decided to make another one. This one is a little different though. It’s a compilation of a few different cuts.

TekTree assembly

Below you’ll find a series of pictures I took while assembling the TekTree.

the baseplate

bottom of the assembly

bottom with box closed

the tree inserted into the base

the arduino and breadboard will just be hanging in the back

TekTree part 1 - TekTree - a christmassy surprise project

Sat, 19 Dec 2015 00:00:00 GMT

I’m very excited to announce a new maker project to you guys ! It will be simple, and thus reproducible, yet fun and quite possibly even useful. (oh, and a tad crazy)

So? What is it?

Well… basically I’m going to assemble and program a little christmas tree thing.

I got the idea after my parents put up their christmas tree. My mom opted for a more “modern” (so I’ve been told) kind of tree. This “modern” tree is basically a home made christmastree-y looking thing bolted together with a few screws. (picture below) At first I didn’t like the idea of a more modern, less authentic, christmas tree but once it was actually up it didn’t look half bad. There were some things missing though. It just didn’t make sense to me to create something new yet be limited by old stuff. For example: my parents put a string of lights around the wooden boards, why not integrate them? Or why not add a dimmer? Or make it follow some flashy patterns? As you might expect by now I am basically going to recreate, with a bit more finesse, my parent’s christmas tree.

a picture of my parents' new christmas tree

Open source?

Of course! All code will be open source! All other stuff (drawings etc.) will also be opened up to the public. Everything can be found on Github once again.

Step 1 - Design

I’ve thought long and hard about what kind of “controller” I wanted to use. And of course I immediately wanted to use a Raspberry pi. But I asked myself, why a pi? Why not use something simpler (like an arduino) to control the lights and switches. After some serious brainstorming I decided to go with an arduino in the end. I figured the Raspberry pi zero would be too cumbersome to work with for a simple christmas tree. Besides, I have a lot more uses for a pi zero than I have for an arduino, and I got more arduino’s laying around than pi zero’s at the moment.

Digital drawing

After deciding to use an arduino I had another big decision to make. Should I follow my parents’ example? Or go a different route? In the end I decided my way was better (kids…) and so I decided to laser cut my wooden designs. One of the other big changes I made is to the base. Whereas my parents opted for something rugged I’m going for a slightly more “finished” look. This means that my base is nice and round with a few fun shapes on it and I even speculated about making the circle sit flush with the ground.(which would mean creating bendable wood! A challenge in and of itself.)

After that came my struggle with the drawing software. I tried creating a drawing in Coreldraw Pro… that was… a painful experience to say the least. Then I tried drawing with a program called InkScape, while better it still didn’t yield any results. Finally I decided to give up and ask for help from a friend.

Luckily my friend Ron Nabuurs is brilliant when it comes to designing a drawing which can be lasercut. So after explaining my idea to Ron he decided to help me out. A picture of what we ended up with can be found below. The source file we used can be found on git.

Designs for the laser cutter

So what now ?

Well… now I have to actually laser cut the pieces and assemble the tree. If we haven’t made any mistakes (or I conveniently forget to mention them to you guys) I should be able to put out another blog this weekend showing you guys both the laser cutting and a test build of the TekTree.

Making things

Sat, 17 Oct 2015 00:00:00 GMT

Even though this week has been extremely busy I wanted to bring you guys a little blog because the things I’ve been doing are truly wicked.

Special assignment

Let’s start with the boring, non maker, stuff. Around 1 pm I received a frantic phone call and a couple of messages from different people about “something”. When I finally got around to dealing with them I discovered something truly awesome.

As some of you know, next semester I will be working on a (school issued) project. This usually involves a group of 4-5 students working on a single big task for about 8 weeks. This time however it will be slightly different, me and 4 other students have been asked to work on a different project than all the other students. Turns out we have been recommended by our teachers and this company, of which we do not know the name yet, has decided to give it a go.

We do not yet know what this special assignment will entail but one thing we know for sure. WE ARE EXCITED ABOUT IT !

Maker thingies

Even though I love software development, deep down I am also still a maker. And a maker likes to “make” things (go figure…), so when I got asked to do a project involving a Raspberry Pi I couldn’t refuse. Even though I had no idea what exactly the project was going to entail I enlisted anyways.

Soon after I had my first meeting with the team and the supervisor, and before long the goal of the project became clear. We were responsible for designing a workshop involving the Raspberry Pi for kids between the age of 12 and 18 years(middle school basically). One of the things we have to do is compare the Raspberry Pi to several other educational boards. This is especially important because schools don’t tend to get anywhere near sufficient resources to even teach programming / IT related stuff, let alone spend their money on fun stuff like a Raspberry pi.

So we came up with “pong”. We are going to create simple controllers (based on potentiometers) and some kind of Raspberry Pi hat to which we can connect them. Besides pong we also speculated about creating our own Raspberry Pi image. I quickly pointed out however that maintaining an image shouldn’t be high on our priority lists and that we might have to think about a different way to solve this problem. What we eventually decided upon was an “Install” script which can be run by either the teacher or some kind of maintainer which will install all dependencies and some example sketches.

The code

Throughout the entire project we will be using a github repo to sync changes. One of the benefits of a github repo, apart from the actual git stuff, is that we automatically share all our code / designs. This means that anyone who is interested can either reproduce or follow along with us.

Enough talking, show us something

Well alright then, we’ve got a couple of things “ready” to show others. The first of which are the design schematics of our controllers and our “hubs”.

fritzing wiring diagram of the controller

We quickly decided upon changing the hub design to a more elegant “hat” (hat being a board you can attach to the Raspberry Pi, not the fedora kind). A render of one such hats can be found below.

render of the raspberry pi hat

We also started working on the controllers. The final design will be compromised of a few “boxes” between which telephone wires run to connect them together. My fantastic soldering / glueing job on the inner workings of a controller and the laser cutted wooden box can be found below.

Back of the controller

bottom of the controller

potentiometer used inside the controller

The front of the controller with the logo

the inside of an assembled controller

The left side of the controller

The right side of the controller

The top of the controller

The laser cutting process

For those interested in the laser cutting process I have also prepared a video:

The end

Well that’s all for today folks, I’ll keep you updated!

I got a new thing !!!! ~ Raspberry pi 7" official touchscreen

Fri, 25 Sep 2015 00:00:00 GMT

Hey guys, as some of you know I was really, and I mean really, excited when they announced the official raspberry pi screen. Now why is that might you ask? Everything the raspberry pi people have released so far has been really solid. The boards, the sd cards, the wifi-dongle, all of them work absolutely fantastic! So as soon as I saw that the official screen was available for pre-order in the UK I ordered myself one and started waiting.

What’s in the box?

First up is the box itself:

picture of the official Raspberry Pi 7" touchscreen display

The box resembles the original packaging from a standalone raspberry pi which is a nice little touch. The picture on the box does not however resemble the pi screen at all which I personally kinda like but it could possibly be seen as false advertisement.

As soon as I opened the box I found a little booklet which contained some health and safety regulations. I have not taken a picture of the booklet because… well it’s a booklet…

Lifting the first bit of red foam reveals yet another piece of red foam in which several electronic bits are embedded. I did take a picture this time:

neatly packed pcb and accessories

From top left to bottom right we’ve got:

4 silver stand-offs
The logic board for both touch and display
Flat ribbon cable to connect the logic board to the pi
4 cables to (optionally) connect the pi to the logic board. (more on this later)

After carefully extracting all the components I yanked out the red foam and found what I was really looking for. The official pi screen nicely protected in bubble wrap (the only way a display should be packaged).

finally! A screen.

After emptying the box I laid out all the pieces and checked whether my set was complete. (it was) In the picture below you’ll see all the contents of the original raspberry pi 7” touchscreen laying next to a raspberry pi B v2. Looking good so far !

all components inside the box

Assembly was easy and straightforward. The only thing I struggled with was the stand-offs. The stand-offs connect the board directly to the metal back of the screen, something which would fry an ordinary pc motherboard. Instead of monkeying around (like usual) I decided to seek guidance and googled a bit. Being one of the early adopters comes at a price though. There was hardly any information on the screen, let alone assembly instructions. I’m sure that by the time I upload this post there is plenty of information about this particular subject but in case you are interested. Yes. You do screw the pi directly into the back of the screen with the included standoff screws.

My case

I bought my pi at Pimoroni and I ordered the case/stand there as well. The exact case I ordered is here. The case is a laser cut layer based case. This means the individual parts of the case are actually flimsy but all bolted together it forms a nice solid block. After applying the case to my screen it looked like this:

fully assembled and ready to hold the screen!

As you can see there is plenty of room for the logic board and the flat cable is dangling freely. One fun thing to note are those plastic screws (which over tighten and break easily, hence why they included a spare screw).

Attaching the pi

The pi itself is connected to the logic board by screwing it onto the stand-offs. Quick tip: connect both ribbon cables before you connect the pi! After attaching the pi I ended up with the following result:

don't mind the wires!

The red and black cables provide power (and ground) to the pi from the logic board. This means that you can power the entire thing (pi + screen) with a single micro USB cable. Neat for those that don’t have a spare micro USB cable. You could also connect the pi with an ordinary micro USB cable to the logic board since it has a USB port. Connecting the pi separately might be required to use some high power GPIO peripherals.

Booting it up

The moment had arrived, I plugged the pi into a battery pack (creating a makeshift tablet-like device) and waited. First boot wasn’t all that promising… Touch didn’t work and I had an annoying little square in the top right. This was however not my first rodeo with a pi and I quickly worked out that the little square indicates insufficient input power. I did a quick update and after rebooting touch worked wonders! The only thing my makeshift tablet test was missing was some sort of input method for the keyboard. For this I used a program called “matchbox”, which was available in the default repository. After playing around with touch for a bit I switched to a solid 2 amp power supply and the little, coloured square disappeared, as suspected.

That’s nice and all… But we want a video

Of course I prepared a little video, wouldn’t be a showcase if I hadn’t now would it. So here goes:

In closing

So, conclusion time. I really like the pi screen, I have been using it for a while now (hence the delayed release of this blog.) and it hasn’t failed me yet. The screen is solid and bright enough for my use case and the touch works wonders. The price point is a little high (88 euro’s) but I suspect the price to go down quite a bit when it becomes available in stores in the Netherlands. That’s it for today’s blog and I’ll see you all later !

What I've been doing

Fri, 11 Sep 2015 00:00:00 GMT

My goals

Last blog we talked about a couple of goals I set for myself. As usual, not all of these have been completed. But instead of focusing on the ones I didn’t complete I’ll focus on the ones I did.

Finish separating my websites

I’m almost done with this goal. The content has been separated but the new portfolio isn’t up yet. I hope to be uploading the new portfolio at the same time as this blog but I can’t be sure of that. I can start uploading the portfolio as soon as I have finished updating a couple of “projects” under the school category. After that the portfolio will be filled with recent work and fully functional. (and quite possibly even pretty).

Clean all my servers (both soft and hardware)

I went above and beyond on this one. I even cleaned my regular pc (soft + hard) and my laptop (soft only). Cleaning my regular (dev) pc wasn’t by choice however. A lightning strike wiped out some electrical stuff and one of the things that got fried was my motherboard. While replacing my motherboard I decided to clean the entire pc since it was open already. After installing a new motherboard I had thought I would have to reinstall Windows. This however was not the case! Windows 8 and Windows 10 both successfully booted after changing my motherboard! This is the first time I have seen any Windows version do that and I was happy to see they’ve improved. I still decided to wipe the system however and replace the ssd Windows was on. A couple of days later I also wiped the linux sdd and replaced that with my old Windows ssd (which was purchased later and thus much faster).

As far as servers go… I have cleaned them too ! Not only have I cleaned all my servers I have also cleaned up both my Dropbox and my Google Drive folders.

Finish my portfolio

I kinda made this goal… kinda. Hear me out on this one ! When I wrote my last blog post I had an idea in mind for the portfolio. There was already a sorting mechanism in place and things didn’t look half bad either. However… I decided to change things around and use a different sorting mechanism (mixitup) and also revamped the look of the portfolio. /images/videos are now the first thing you’ll get to see and they will be as wide as the page itself. (some videos , like app showcases, will have black borders though). Directly beneath the media content is the customer and designer information and below that you’ll find a short detailed summary for each aspect of the project. So I decided to chalk this one up as a success because I’ve done more work than I had originally anticipated :).

The other goals

Only a few of the goals I had set I haven’t reached. I have given up on losing weight and because of the extra portfolio work I didn’t really start LSR either. 5 new tutorials is a detable goal. I haven’t exactly written 5 tutorials but I have done some work on mi-soft.nl (and still wrote 2 tutorials). And 1 blogpost… well I’m trying !

Losing some of my stuff

As I mentioned before I have lost some of my hardware to a lightning strike. I didn’t just lose my motherboard however. I also lost my microphone (blue snowball), my keyboard (razer blackwidow), my xbox controller receiver, 8gb’s of ram and some random usb peripherals. Some of the stuff was insured and thus I have gotten some money for it. However I did not simply want to replace the lost stuff, I wanted to upgrade too.

I upgraded my keyboard to a ducky shine with black cherry mx switches. I have to say I’m still getting used to the linear-ness of the switches but so far so good. The big upgrade however is my recording setup. I went from having a single blue snowball to a blue yeti with shock mount, pop filter and boom arm. Initial tests prove to be very promising and there is a definite upgrade in sound quality. Aside from sounding a bit better it looks A LOT better. It almost look’s pro! Here, have a look:

Looking spiffy!

Don't mind the mess, it's the mic we're looking at

Youtube channel split + new intros and outros

Mi-Soft.nl and Rick van Lieshout are now 2 separate Youtube channels. Mi-Soft.nl will only be used to host tutorials. Nothing else. My personal channel (Rick van Lieshout) will host all other things.

I have designed a new outro for all of the Mi-Soft.nl video’s. I made the outro with “Camtasia studio” and “Gimp”. The outro can be found here:

I also bought some intro’s from Flixpress. Both of these can be found here:

New look, new comments, new everything!

Tue, 16 Jun 2015 00:00:00 GMT

Separating the personal stuff

First off, it’s been a while… I know. ☺

I’m trying to separate mi-soft.nl and myself, yet keep them together somehow. I have removed almost all personal content from mi-soft.nl and am currently busy moving it to this “personal website”. The end goal is to have them completely separate apart from the “software” tab. Software that I want to reveal to the public (more so than some scripts I use myself) will still be uploaded to mi-soft.nl and more personal projects (like CSVtoDatabase) on my github.

In order to “keep the sites together” I have given this site the exact same look as mi-soft.nl except for the colours. I have also made a little system that allows users to switch the website’s language. Everything apart from the blog will be Multilingual. Doing all these upgrades however also means upgrading Disqus, which meant losing all my comments. While sad, I feel like it had to be done.

Replacing the “about me” page on mi-soft.nl

I will be replacing the about me page on mi-soft.nl with a “Linux software review” (LSR for short) section. LSR will consist of short video’s (in Dutch) promoting Linux software.

Another maker project

I am currently gathering materials and setting up the software for another cool maker project. It is going to include both the arduino and the raspberry pi. But that is pretty much all the information I want to disclose right now.

School

I have only 2 days of school left this year, (and maybe a ceremony) which means an increase in productivity (hopefully). There are a couple of things I want to do before school starts again:

Lose 20+ pounds
Finish separating my websites
Clean all my servers (both soft and hardware)
Finish my portfolio (in both Dutch and English)
Get started with LSR
Write at least 5 new tutorials on mi-soft.nl
Write at least 2 new blogs. (not counting this one)

Let’s hope I can get all my planned projects done and then some.

A status update

Sun, 16 Nov 2014 00:00:00 GMT

School

In my last blog post I mentioned that I was busy with schoolwork (and all related matters like friends & gaming). That all changed yesterday, I decided not to game as much anymore and focus on something exciting again. I have no idea what kind of exciting thing I’m going to focus myself on, but most likely it will be either Arduino or Raspberry PI (cluster / solo).

Some more great news! We were able to collect 15 points (out of 60 in a year) during this first semester. And I got them all. The most difficult test for me was of course the Dutch language test. Which I barely passed. One of the other “harder” tests was a subject called “UML”, and as some of you might know… I hate, most of, UML.

A new semester has started this week and we’ve already had a taste of what is to come. Sadly what is to come isn’t all that Linux friendly. They have opted to use MsSQL instead of the (much better) MySQL(MariaDB) / PostgeSQL. This means that I, and another classmate, have to install virtualbox with Windows on our laptops. Shouldn’t be a big problem should it? Well no it shouldn’t. But me trying to be smart resulted in me downloading a “stripped down / light” version of Windows. (as if there is such a thing ! ha!) After setting up Windows, and installing office (since we’ll be using Access (another YUK!) I tried installing MsSQL. Turns out however that one of the stripped features was powershell, and for some inexplicable reason MsSQL will not install without powershell. To make matters worse Microsoft decided that they wouldn’t offer an install package through the website anymore since powershell is built-in in the newer versions of Windows. So I had to resort to giving up almost 30GB of hard drive space just to install MsSQL… (and I will probably end up using MySQL anyways)

Numix-icon-theme-square

Last night a friend of mine tried installing the Numix square theme, something that usually doesn’t require my help. Last night however my friend did need some help because the theme just would NOT install correctly. (kept throwing a GPG key error) So after trying to install an older package (which I did not know he removed from his system) I finally decided to google the matter. Turns out the “Square” theme’s actually cost money (who knew?). Turns out that Antergos Linux has some sort of a license to use them but general Linux does not.

That is why the team behind Numix has requested the package to be removed from the aur. And their request was granted. (as it should have been). Last night however we didn’t have the tools handy to be able to make a purchase of the Icon theme, so I sent him my version and he installed that instead. I will however buy a license from them today, since I use the icon theme on pretty much all my machines.

Arduino sneak peak

I acquired some new toys for the arduino, two of them being the “Adafruit mini 8x8 LED matrix”. To use them I had to install 2 libraries, and with those came “Example sketches”. One of those example sketches was a “Robotic eye”, naturally with me having 2 new toys I decided to create a little “face” (just eyes) with the prebuilt sketches. Anyways I thought that looked cool so I made a photo which you can view below:

Google eyes... but matrix style

Me, my blog, and projects.

Tue, 21 Oct 2014 00:00:00 GMT

The blog

So, I’ve already gone past my weekly schedule. Which tends to happen every time… I’m not giving up on the blog though ! I’ve just been far too busy. Busy with what you ask? well… college. Now surely the first period shouldn’t be that hard for someone who can code reasonably well already You’d be right in thinking that, but.. that’s not the only part of college, college also means new friends. New friends mean gaming. So yeah I’ve been gaming a bunch. Sorry.

Projects

For now I’m putting off some of the projects I had originally planned. I simply can’t find the time to write detailed blog posts on them and I don’t want to disappoint you guys with half-baked posts.

I did start a new “project”, meaning I ordered 2 of the most beautiful computers on the planet. Yes, I bought 2 iMac G4’s . Even though I don’t like Mac os, or the designs Apple is currently pushing out, I do really like these older Mac’s. I’ve ordered 2 of them from the internet, the seller is currently looking for a box to send them both in since they are pretty big and heavy. I can only wait and hope they will arrive in good condition.

So I hope you guys appreciate this little update and can forgive my absence. Have a nice day!

Shellshock and why you shouldn't get your panties in a bunch.

Sat, 27 Sep 2014 23:00:00 GMT

On September 24 2014, A Bash vulnerability, known as “Shellshock” was disclosed. This “bug” allows an attacker to run code on your machine, which is bad, very bad.

Now then, what exactly is “Shellshock”. Shellshock is a “bug” (or vulnerability) that can be used to remotely run code on someone’s machine by altering an environment variable. The real danger however lies in programs that can alter these variables without requiring root. Thus enabling anyone with access to your computer to run commands. Imagine someone attacking you twice, the first time they’ll create a user and the second time they’ll set a password for the user. After that they can just log in to your system and destroy the entire thing, or worse still, monitor your every move. Some examples of the services which can alter environment variables without root privileges are:

OpenSSH (which 90% of all Linux distros use)*
DHCP clients
Apache servers with CGI scripts

When are you vulnerable?

This seems to be a point many news sites forget about. When exactly are you vulnerable and how worried should you be. The thing is, in order for you to be vulnerable you’d have to have one of two things:

A server which is open to the world (web,vps, etc.)
An intruder in your network.

Most people won’t have either one of those things. And to be fair, if you can’t trust the people on your own network you’ve got far bigger problems than “Shellshock”. Unless of course you’re using a whole lot of “free-wifi” services, in which case you really don’t know who is on the network.

I’d imagine the number of people having a “server” , or simply a computer which is accessible from outside your home network is way higher. So if you are one of those people, please patch your bash to avoid any trouble. ( I too like to have an easy way to log into my server from anywhere (work / family / vacation) and thus I would be vulnerable.)

How to test whether you’re vulnerable

Testing whether you’re vulnerable is quite easy. You can just use this “attack” on yourself with an “echo” command (which won’t cause any harm). If it works, you’re vulnerable. If it doesn’t you’re not. Here’s how to construct a test command:

env x='() { :;}; echo vulnerable' bash -c 'echo hello'

If you’re not vulnerable you’ll get:

bash: warning: x: ignoring function definition attempt bash: error importing function definition for `x' hello

You might also simply get “hello”. If you are vulnerable however you’ll get:

vulnerable hello

How to patch your system

By now most distributions have updated their repo to include the patched version of bash. If that is the case you can simply do a regular update:

Ubuntu / re-spins

sudo apt-get update && sudo apt-get upgrade

Arch / re-spins

sudo pacman -Syu

If however you’re using a mac you’ll have to do it by hand. Since Apple hasn’t yet released an update nor commented on the matter. Instructions for a mac can be found here.

In summary

So should you worry? Yes and no. If your computer (/server) is exposed to an outside network you should absolutely worry. If it’s not however you are relatively safe. Still, patching is a simple and fast process, so just patch it and get on with your life !

Documentation

For those of you who like to research this “Shellshock” vulnerability some more, here are 2 entries in the NVD: CVE-2014-6271 & CVE-2014-7169.

If you want to see Shellshock in action I recommend watching the following video from Quidsup:

The pi cluster, home automation & kickstarter monitoring

Sat, 27 Sep 2014 00:00:00 GMT

Some of you might be wondering “why doesn’t he write more blogs about the cluster?” and I wouldn’t blame you. I have a pretty good reason for not blogging about the pi cluster (even though I promised sample code). That reason is “home automation”. As some of you might know I’m big into home automation, and have done quite a bit already. But it’s always fun to try something different!

I usually use single or double relay boards which you can control with an arduino or a Raspberry pi. This time however I ordered an 8 relay board instead of a double relay board. I got the board from China at a store called “Sainsmart”, here’s the link to the exact board I got: https://www.sainsmart.com/products/8-channel-5v-relay-module

So what does that have to do with the cluster you ask? Well, I used to have a Raspberry pi to experiment with… But I used that in the cluster to save some money. (here I was, thinking to be smart!) So I had to “disassemble”(taking the top off) the cluster to connect the relay board. In the next blog-post I’ll have a couple of pictures showing off the relay board and the pi cluster.

I donated a pi

While talking on IRC with a friend(Robbie Ferguson) of mine I discovered that he had never used a Raspberry pi before. And to make matters worse he told me he had lost interest in them now. When I asked why he never got one he told me the following:

When they were new I couldn’t get one here in Canada, and now it’s been so long that I’ve lost interest.

While discussing the fact he jokingly told me, if you’ll send me one I’ll try it out. A small problem arose though, me sending a pi to Canada would cost way more than actually buying one over there. So we decided that I’d just send him the money and he would order his own. I can’t wait to see what he does with it. If you guys are interested in seeing what he does with it too you can check out Category5.tv

What Robbie doesn’t know is that I wanted to do something for the show / him anyways. I was originally planning on just donating $20, but giving him the joy that comes from owning a Raspberry pi seemed like a way better plan! Besides, he’s been working really really hard on the new Studio D so he deserves some fun!

Kickstarter monitoring

I have once more decided to back a certain Kickstarter campaign and find myself checking the website a lot. I plan to change this by creating a little script (don’t know which language yet) that will check it for me and report back. More info about this project to come.

Raspberry pi cluster step 5: Expanding the cluster

Fri, 19 Sep 2014 00:00:00 GMT

In today’s post we’ll have a look at expanding the cluster with more nodes! We will have a look at how to clone them and how we can set up a universal login using ssh keys.

Windows users can use the “read” option of the previously installed win32imager to create an img.

Linux

Insert the tested and configured SD card and use lsblk to find out on which /dev/ the SD card resides. Now use dd to make an img of your Raspberry pi master node (note! this can be used as a backup too) with the following command:

sudo dd if=/path/to/sd/card of=/home/`whoami`/raspy.img

After running that command you will have an image of the configured Raspberry pi in your home folder. You will now have to write this image to all SD cards.

After writing all the images to the SD cards you have to change the hostname on all “copies”. To do this put the SD card into one of the pi’s, let it boot , and log in. Then give it the following command:

sudo raspi-config

Next, you’ll have to change the ip on each of the nodes by changing the “address” line after executing the following command:

sudo nano /etc/network/interfaces

After setting up all 4 pi’s and plugging it all in you should see all 4 led’s light up on the pi’s. If this is not the case you might’ve forgotten to change either the ip address or the hostname. (which will lead to a conflict resulting in one of them not having internet access). Mine now looks like this (note the dramatic night picture taken at 10 past 12 am):

Logging in to the nodes with ssh keys

To make working with the cluster a lot easier we will use ssh keys to login in place of the old password. We are going to generate the master node’s key first, to do this enter the following command.

ssh-keygen -t rsa  -C "pi@raspberrypi0"

You’ll be asked where you want to save the key, the default directory is usually fine. After that you will be asked to enter a passphrase. (the passphrase is optional but also recommended in a production environment) After generating the key a “randomart” image will appear. After generating the key you want to execute the following command:

cat ~/.ssh/id_rsa.pub | ssh pi@192.168.1.8 "mkdir .ssh;cat >> .ssh/authorized_keys"

Keep in mind you might have to change the ip address in the previous command. After executing this command for the 2nd node, repeat for the third,fourth,fifth, etc., etc. nodes.

After we have copied the key to all nodes we have to edit the previously created machine file (on master) to include all the ip addresses of the other pi’s. To do this type in the following command:

nano mpi_scripts/machinefile

Simply add all the other pi’s ip addresses to the file (each on a new line).

Testing

Now that we’ve added the pi’s ip’s to the machinefile we can use them from our master node. To test this we can run the following command:

mpiexec -f mpi_scripts/machinefile -n 4 hostname

This should give you 4 hostnames. If it does you’ve set it all up correctly! We can even try and run some C code with them by running the next command:

mpiexec -f mpi_scripts/machinefile -n 4 ~/mpich_build/examples/cpi

Is that correct? O.o

If you have a look at the screenshot above you’ll see that the cluster as a whole (4 pi’s) actually take longer to calulcate pi than a single node would. This is to be expected with something that runs this quickly. The cluster has to communicate with each of the pi’s, which results in a tiny little bit of network overhead. In the case of calculating pi this network overhead turns out to be greater than the time it takes to calculate pi to 15 digits.

Now that we have a complete (working) cluster we are all set to start using it! But first, here’s a video of me power cycling the cluster:

Raspberry pi cluster Step 4: Setting up the master node

Wed, 17 Sep 2014 00:00:00 GMT

After installing raspbian to the sd card it’s time to plug in that SD card and fire up the Raspberry pi. If everything went well you should end up with a screen similar to mine:

raspi-config boot screen

The first thing you should do is select the “Expand Filesystem” option, this will make sure the OS can access the entire SD card. After doing that you might want to change the user password (user = pi). The last thing you want to do is hidden in the “Advanced options”, after going there you should see:

expand filesystem menu options

In here, select the “A2 Hostname” option and rename it to something better. (I appended a 0, indicating this would be the master pi)

After doing all this hit finish, reboot the pi, and log back into the shell. (default user = pi pass= raspberry) The first thing we are going to do is update the pi, in case any updates have come out since the image was uploaded to raspbian.net. We can update the pi by running the following commands:

sudo apt-get update && sudo apt-get upgrade

After updating the pi we need to give the pi a static ip address (so we can find it later). We can do this by following a couple of steps:

sudo nano /etc/network/interfaces
replace iface eth0 inet dhcp with iface eth0 inet static

Directly underneath add the following lines: (remember to change info if required)

address 192.168.1.11
netmask 255.255.255.0
network 192.168.1.1
broadcast 192.168.1.255
gateway 192.168.1.1

Hit ctrl + x to close nano and hit y to save the changes.
reboot the pi

After that we have to install some required software, we can do so with the following command:

sudo apt-get install gfortran

After installing some of the required software we still have to install “mpich3”, do so by running the following commands:

mkdir /home/pi/mpich3
cd mpich3
wget https://www.mpich.org/static/downloads/3.0.4/mpich-3.0.4.tar.gz
tar xzf mpich-3.0.4.tar.gz
sudo mkdir -p /home/rpimpi/mpich3-install
sudo mkdir -p /home/pi/mpich_build
cd /home/pi/mpich_build
sudo /home/pi/mpich3/mpich-3.0.4/configure -prefix=/home/rpimpi/mpich3-install
sudo make && sudo make install

After installing mpich3 we need to edit a file. Execute the following command:

sudo nano /home/pi/.profile

Scroll down all the way and add the following text:

PATH="$PATH:/home/rpimpi/mpich3-install/bin"

After adding that line hit ctrl + x and save your work. Then either restart the shell or reboot the pi.

After you have rebooted the pi you have to create a so-called “machine file”, we will store this file in the folder “mpi_scripts” in our home directory. To do this execute the following commands:

mkdir ~/mpi_scripts
cd ~/mpi_scripts
echo `/sbin/ifconfig eth0 | grep "inet addr" | awk -F: '{print $2}' | awk '{print $1}'` > machinefile

The last command creates the machinefile and populates it with the ip of this (master) node automatically. With all of this setup we can do our first real test with the mpi commands, so try out the following command:

mpiexec -f machinefile -n 1 hostname

The -f is the filename parameter (so you would type the machinefile path behind that). The -n is the number of nodes we want to use (we are working from 1 now). And finally “hostname” is the command.

if all went well you should now see the hostname of the raspberry pi (for me: raspberrypi0). After this we are done setting up the master (for now) and can begin copying the SD card. To shutdown the pi you can execute the following command:

sudo shutdown -h now

Raspberry pi cluster Step 3: Installing the OS

Sun, 14 Sep 2014 00:00:00 GMT

Installing an OS for a Raspberry pi is quite different from installing an OS on any other computer. We will have to write an “image” to the SD card rather than actually “installing” anything. Don’t worry if you’re confused just now, even I don’t know what I’m rambling on about at times!

Linux

First off, if you want to use Windows, skip this bit by clicking here.

First we will have to download the .img from the Raspbian website, we can do so by clicking here. After downloading the image we can use a kernel application known as “dd” to write the file to our SD card. We will first need to find out which /dev/ the SD card is currently using. To do this we can use the following command:

lsblk

This will show us a list of all the connected volumes. You should be able to determine which of the entries is your SD card by looking at the size of each entry. If you are not able to determine which one you need based on the size of the entry however, you can try removing the sd card and re-running the command. The entry that disappears is the SD card.

As you can see below, /dev/sde is my SD card

NAME   MAJ:MIN RM   SIZE
sda      8:0    0 119.2G
├─sda1   8:1    0   243M
├─sda2   8:2    0   7.5G
└─sda3   8:3    0 111.6G
sdb      8:16   0 111.8G
└─sdb1   8:17   0 111.8G
sdc      8:32   0 111.8G
├─sdc1   8:33   0   350M
└─sdc2   8:34   0 111.5G
sde      8:64   1   7.5G
└─sde1   8:65   1   7.5G

Now that we know both the input path (the downloaded iso’s location) and the output path (the /dev/sd* device) we can assemble our dd command as follows:

dd if=/path/to/iso of=/path/to/sd/card

When you execute the dd command the terminal will appear “busy” for the duration of the write. You can either wait patiently until it’s done or check up on its status by entering the command:

sudo kill -USR1 `pidof dd`

Windows

If you ended up here you have chosen to use the much inferior Windows to get Linux on your Raspberry pi’s. Don’t worry though, you can still install the, much superior, Linux images to your SD cards.

The very first thing we have to do is download the Raspbian image, which we can do by clicking here. After downloading the image we will have to download some software to write the image to the SD card. The tool we are going to use is called win32DiskImager, and can be downloaded free of charge right here.

After the tool has been downloaded (and installed) we have to start it by double clicking on the .exe file, which will leave us staring at the following tool:

win32diskimage

Using the image above as a reference:

This is where you have to specify the image location (you can hit the blue box to select a location)
Here you will have to specify which device you want to write to.

After keying in the right values you can hit “write” and wait for the “install” to finish.

First boot

Once you boot up the pi you will get a settings screen, you can go through it if you wish. (change password and so on) If you want to start the GUI you can type:

startx

Raspberry pi cluster Step 2: Assembling the hardware

Wed, 10 Sep 2014 00:00:00 GMT

So today I finally received the required parts to finish up my Raspberry pi cluster and I was extremely excited! To share this excitement with you lot here’s a picture of all the parts I’ll be using:

all the bits!

I began by building the Raspberry pi “tower”, sadly I forgot to take a picture with the DSLR so I only have this one cell image to show you guys:

rising up!

On the picture above you can clearly see how the “tower” is being assembled. I’m basically using the stand-offs that come included to mount the pi to the bottom plates and then instead of attaching a top plate I attach a bottom plate again. I did have to customize the plates a bit to prevent the pi’s from overheating (because there now is pretty much no room between a pi and the top plate).

After mounting all 4, the “tower” now looks like this: (oh, I also added SD cards)

Assembled tower of pis

Then I attempted to glue the tower to my switch… Some complications came up though. First I glued my knife to my finger and then my finger to another finger. Once I had successfully managed to glue the tower onto the switch I needed something to apply pressure to let the glue dry. Since I didn’t have anything heavy handy I had to improvise. And so I did, in true nerd style:

balancing a weight.... a netbook

Yes, those are 2 packs of cards (which weren’t heavy enough) and an old netbook. I went to have dinner while I let the glue dry and when I returned everything was all glued firmly into place.

After adding the cables and the usb switch I am left with my beautiful raspberry pi cluster:

looking great!

In my next blog post, I will cover how to install raspbian on a raspberry pi in general and how I plan on cloning the install across all pi’s

Raspberry pi cluster Step 1: Shopping

Fri, 05 Sep 2014 23:00:00 GMT

So, A raspberry pi cluster. What is it and what does it do?

Well, a raspberry pi is a small $35 dollar Linux computer with an SD card as a hard drive and an USB cable as it’s power source. While they are only $35, they perform really well!

A cluster is basically a group of computers that work together to run a very intensive program. Say for example you needed to calculate 1000 things with a single raspberry pi and every calculation takes 1 second. That would be 1000 seconds! Now if you add a second raspberry pi and you have them share the load they will be twice as fast! Imagine running 4 of them, that would cut it down to 250 seconds! Well that’s exactly what I plan on doing

What software are you going to use?

I am going to use raspbian for the OS and I will be using python (and mpi4py) to program the cluster.

What did you need and how much did it cost

Below you will find a list of items you would need if you were to build a 4 pi cluster from scratch. Prices are in Euro’s because that’s what I pay with (I know, crazy right?)

4 raspberry pis ( €131.80 )
4 ethernet cables ( €3.40 )
4 USB power cables ( €7.20 )
4 sd cards ( €12.72 )
4 raspberry pi cases ( €11.72 )
An Ethernet hub ( €10.95 )
An USB hub ( €2.64 )

This all adds up to ( €180.43 which is $233.76 according to google)

I ordered some of the stuff from a shop in china called ”banggood”. Since they deliver all around the world I will add the links to those items below.

The raspberry pi cases can be found here
The 8gb SD cards can be found here
The USB hub can be found here
The USB power cables can be found here

My brand new blog

Fri, 05 Sep 2014 22:00:00 GMT

Hey all and welcome to my new blog! A couple of questions pop-up right away I’m sure. So let’s get those out of the way right now!

English?!? Is this a joke?

Nope, I have decided to blog in English for a couple of reasons.

It will be easier for me to write articles.
I get to share my posts with a couple of non-Dutch friends.

Why did you start over again?

Yeah, I know… This is like the fourth time I start over isn’t it? Well let me give you some reasons for switching:

Control. I now have full control over how the page looks and performs.
No more Wordpress!!! Many of you will know I have a love/hate relationship with Wordpress. But in the end, the hate won over the love. I am just fed up with the entire system and its problems.
Because I wanted to convert to English. And why not convert platform at the same time?

So what can we expect?

I’ll be doing a lot more technical blogs. (up next: raspberry pi cluster) I also plan on doing some code examples and general rambling. So pretty much the same as any other blog.