What I learned in my first year at AutoScout24
Martin Lechner · October 4, 2018 · 7 min read
Disclaimer: It is impossible to write down what I learned in one year. This write down is therefore focussed on some of the bigger topics. I work at AutoScout24 as Senior Software Engineer in the dealer segment so this might be a bit biased towards my employer. With that out of place, let’s start.
The people
Coming from a solid German medium-sized business where the majority of people speak German, it was quite interesting to dive into a very international company with co-workers being from around the globe and English being the main language at my day job. I met and worked with people from Brasil, India, Ukraine, Russia, Canada and nearly all of the EU amongst many other countries. This has shown me many different cultures and mentalities. In the end all of those people are very cooperative and do their best to drive the company forward.
Some of them inspired me to push even harder and also look further beyond the technical aspects of my job. I really feel that picking up soft-skills and leadership knowledge has helped me doing a better job from the day I started getting into it. And yet I’m still at the very beginning of this non-technical journey.
I highly recommend reading Kim Scott’s Radical Candor. Even if you don’t really lead people this can improve your mindset regarding interactions with your co-workers tremendously. I started seeing things that were invisible to me before.
Functional programming in Scala
When I left my former employer I was keen on learning more advanced Scala. My key learning materials this year have been “Advanced Scala with Cats” and “Functional Programming for Mortals”, which I can highly recommend to anyone trying to get their feet wet in functional programming.
Functional Programming resolves around a set of ideas to make your programs more robust and maintainable.
The most important three are:
Purity — This means that your functions should not have side effects. It becomes much easier to reason about your code and also leads to easier concurrency handling.
Immutability — This means that things shouldn’t be modified. Your functions will return new objects or values but do not mutate stuff or use things like getters and setters found in OOP.
Composition — Compose your program out of smaller functions to have easier to grasp and reusable pieces.
To make Scala even more functional programming oriented, it can be helpful to use a library like Cats. Cats itself provides many useful abstractions, with many of them heavily inspired by Haskell.
There is also the so-called Typelevel whole ecosystem evolving around Cats as its core. It provides pure side effects (cats-effect), JSON handling (circe), JDBC (doobie), Http (Http4s), lenses (Monocle), streaming (FS2) and refinement types (refined) amongst many other things.
This gives the functional programmer a strong toolbox to solve every day problems in a purely functional way. You might ask: The real world has side effects, how does this play together with purity?
We’re using the solution of making the side effects explicit and wrapped in a lazy evaluated structure by using for example a Monix Task.
This lets you split the specification of the application’s behaviour from the execution and preserves referential transparency (compared to e.g. Future from Scala standard library, which is eagerly evaluated).
For general application architecture we try to use tagless final to create DSLs which covers the building blocks of the application and splitting it from the implementation. That usually leads to easily readable programs because the specification of a program is usually relatively simple compared to the implementation details.
Having Scala as one of the primary languages at our company, this also opened the possibility to attend ScalaDays Berlin where my colleagues and I had the chance to listen to awesome talks and have inspiring discussions with people from around the globe. I really recommend going to conferences and meeting like-minded people.
It has been an interesting journey to become a more FP centric Scala developer by learning much of the concepts mentioned above while being productive and also putting them into production when the opportunity had arisen.
A big cloud environment
Scout24 has the biggest cloud infrastructure I’ve ever seen. One of the central architectural principles is “AWS first”, which means that native cloud services should be used if possible. DynamoDB, Kinesis, Lambda, CloudFormation, SNS, SQS, Lex, Aurora, …
We’re using all of them and I literally started at zero one year ago. Now I’m feeling pretty proficient in choosing the right service to do the job and have probably also produced hundreds of lines of CloudFormation templates to automate cloud infrastructure deployment of our services.
Which directly leads to “You build it, you run it.” When we develop a service, it is our duty afterwards to keep it running and to make sure that it fulfills its purpose. There is heavy use of DataDog for metrics, Opsgenie for alerting and the ELK stack for logging. After diving into those topics, I began to realize that I knew close to nothing about DevOps culture. With everything those tools provide to us, one really has to think about using them in a reasonable way so that they provide insights and help you recover in case of an incident.
Having continuous delivery with use of feature toggles was also pretty new to me coming from a more project centric background. But be warned: when you experience it, you might never want to go back.
Organizational challenges
As you might already have figured out right now, AutoScout24 is a pretty big organization which comes with its own challenges. If you look onto the detail page of a car listing, you see many different kinds of information shown by various teams from different market segments. Coordinating interactions and dependencies between teams is not an easy task and there is no silver bullet solution to that.
We recently started using OKRs as a mean to sync goals between segments and having taken part in an internal workshop helped tremendously grasping this topic. I strongly feel that this will help our organization to overcome some of the major challenges that lie ahead of us.
Microservices
To provide the teams with maximum autonomy we are using microservices, which are small services focusing on a certain part of the domain (see bounded contexts in Domain Driven Design for more information). They usually provide synchronous apis, with some of them also using Kafka to work asynchronously.
You might ask how a complex page like the detail page is filled with information from small services.
For the frontend part there is UI composition in place. Our services serve html fragments as one of their interfaces and there is a service based on nginx to stitch them together into a page. It is used on most of the pages and also scales pretty well to the amount of our traffic.
The second movement I’m experiencing right now is building pages in React and using GraphQL endpoints to feed them with data. This is certainly easier when your team owns the whole page, but not so on complex ones. There is usually a backend for frontend, which provides the GraphQL api and aggregates data from different downstream services.
Verdict
I really enjoyed this year, got to know many inspiring people and could pick up so much valuable learning from many different sources. I feel that our current stack with microservices on AWS, mostly written in Scala, is working pretty well for us. Adding a pure functional core has led to decreased bug density and makes us move faster.
If you’d like to know more, have a discussion or even join my journey, I’m more than happy for any input.
