I spent the last two years working as a junior full stack engineer, mostly Spring Boot and a relational database behind it, plus whatever frontend the job needed that week. Now I'm job hunting again, and going through job postings made one gap pretty obvious pretty fast: almost every listing wanted Docker and Kubernetes experience, and I had never touched either one. My actual job never had a reason to. We shipped one app to one server and that was that.
So instead of padding my resume with a word I couldn't back up in an interview, I decided to build something real enough that I'd actually have to learn it properly. That became AccessBridge, a project modeled loosely on how companies like Imprivata do unified access management: you log in once, and from there you get short lived, audited access into a bunch of different internal apps, with everything you do getting logged and watched for anything weird. I picked something more ambitious than a single app on purpose, because a system made of several services that actually have to talk to each other is a lot closer to what these job postings are describing than another CRUD app would have been.
On paper it's five backend services plus a dashboard. One service owns who you are. Another owns what you're currently allowed to touch and for how long. Every interesting thing that happens gets pushed onto a Kafka topic. One service reads that stream and keeps a permanent audit log. Another service reads the exact same stream and looks for suspicious patterns, like the same account logging in from two different countries within an hour.
I split the build into six phases and refused to let myself jump ahead until each one actually worked end to end. That turned out to be the single best decision I made, mostly because of what happened in phase four.
The bug that made microservices click for me
By phase three I had audit-service up and consuming everything off the Kafka topic, deserializing each message into its own event type and writing it to an append only table. It worked great. Then in phase four I built anomaly-service, which also needed to read that same stream, and once it detected something suspicious, it published a brand new event type back onto the topic so the anomaly would also show up in the permanent audit trail.
The moment I shipped that, audit-service, which had been running fine for a whole phase already, would have started throwing errors on every single new anomaly event, because its deserializer had no idea what to do with an event type it had never seen before. In a single service app this problem just doesn't exist. There's no such thing as "a message you don't recognize" when you're the only one writing and reading your own data.
The fix was to wrap the Kafka deserializer in an ErrorHandlingDeserializer so an unrecognized event type gets logged and skipped instead of stalling the whole partition, and to make sure any service consuming the full stream keeps its own enum of every event type that could possibly show up, not just the ones its original author knew about at the time. It sounds obvious writing it out now, but living through it was the first time I really got why people talk about designing for forward compatibility between services instead of just designing the happy path. Nobody tells you this in a tutorial because a tutorial doesn't have a second service that ships six weeks after the first one.
CORS reminded me the browser is not curl
Up through phase four every single caller of these services was either curl or another backend service, so I never had to think about CORS at all. Then in phase five I built a React dashboard that calls all five services directly from the browser instead of going through some kind of gateway, and suddenly requests that worked perfectly in Postman were getting blocked before they even reached my Spring Security config.
It's a small thing, but it was a good reminder that "it works when I test it" quietly depends on how you're testing it. Same origin server to server calls and a browser making cross origin requests are not the same trust boundary, and I'd genuinely never had to care about that distinction before because nothing in my previous projects ever crossed it.
Kubernetes found two bugs docker-compose never could
Everything above ran on docker-compose for the first five phases, which was already new ground for me since Docker had never really come up at my old job either. By phase six I was feeling pretty good about myself, docker-compose had been solid the whole way through. Then I got to the actual reason I started this project in the first place, deploying the thing to a real local Kubernetes cluster with kind instead of just writing yaml files and calling it done, and Kubernetes immediately found two bugs that had been sitting there the whole time.
The first one was genuinely confusing to debug. My Kafka pod kept crashing on startup with no useful error message, right after a log line that just said something like "port is deprecated." It turned out Kubernetes automatically injects environment variables for every Service into every pod in the namespace, so a Service named kafka meant every pod got a KAFKA_PORT variable whether it wanted one or not. The Kafka image treats any environment variable starting with KAFKA_ as broker configuration, so it happily ingested that bogus injected value and choked on it before Kafka itself ever started. I only found this by running the exact same container with docker run and the exact same environment variables outside of Kubernetes, watching it work fine, and realizing the only difference left was whatever Kubernetes itself was adding. The fix is one line, enableServiceLinks set to false, and I added it to every deployment once I understood what caused it.
The second bug was more of a resource problem than a code problem. With five JVM services all cold starting at once on a laptop sized cluster, Spring Boot's own startup was taking close to a minute under the CPU contention, and my liveness probe's default delay was way shorter than that, so Kubernetes kept killing containers that were just slow, not broken, over and over. The fix there was a startupProbe with a generous timeout that gates the liveness and readiness probes until the app has actually had a real chance to come up. Neither of these bugs would have ever shown up if I'd stopped at "the yaml builds without errors," which is exactly what I would have done if I hadn't decided to actually run it for real.
The part that had nothing to do with code
Somewhere in the middle of all this my laptop's disk hit basically zero free space, which I only noticed because a background terminal command failed with a disk full error. It had nothing to do with the project, it was just years of downloads and cache buildup finally catching up with me, but it was a good reminder that the boring infrastructure stuff, disk space, memory limits, how much CPU a VM actually has, matters just as much as the code you're proud of. You can write a perfectly correct Kubernetes deployment and still watch it fail because the machine underneath it ran out of room.
What I'd tell someone starting something like this
Build it in phases and actually finish each one before moving to the next, even when it's tempting to sketch out everything at once. Most of the real lessons here only showed up because a later phase had to coexist with a decision I'd already shipped and couldn't quietly go back and redo. And when you get to the deployment part, actually deploy it somewhere, even somewhere small and local, instead of stopping once the config file looks right. The gap between "this yaml is syntactically valid" and "this actually runs" is where basically everything I learned in phase six was hiding.
I still haven't deployed this for real against AWS, mostly because EKS, RDS and MSK all together cost real money to leave running just to prove a point, so for now the Terraform and the AWS specific Kubernetes overlay are validated but untested against an actual account. That's the honest state of it, and I'd rather say that plainly than pretend otherwise.
I can't put "two years of Kubernetes experience" on my resume because of one project, and I'm not going to pretend I can. But I can say I've actually deployed a multi service system to a real cluster, actually hit the kind of bugs that only show up once containers are talking to each other over a real network instead of docker-compose's DNS, and actually fixed them by reading logs and reasoning about it instead of copying a Stack Overflow answer. If an interview gets into the weeds on any of this, I'll have something real to say back, and that's the whole reason I built it.