Event driven microservices

I’ve got lucky I got my spot in last London GOTO conference. Theme was obviously event driven microservices and how they simplify whole architecture of complex applications. After 2 days of listening about them there was no other choice than prototype.

Most intriguing for me was the idea of performing read through event bus as well as writes. It kinda hit me as being against CQS. Couldn’t get why and how. Being in mindset of transactional storage wasn’t helping as well :P

Building event driven microservices

Requirement as simple. One service is providing data and other is requesting and processing it. And some interface to interact with. No need for web, as it’s prototype/proof of concept. Console app it is and Symofny as framework of choice as the simplest way to scaffold code. Yesterday I was thinking about getting kafka but again, I wanted to see results not caring much about technology.

If you want to do system like that in real life please don’t use PHP for that :D

Code you will find here.

So how it works?

I’ve got positively surprised by the effect. Everything was obviously done under a second. Even checking on microsecond level shown loads of time saved in comparison to HTTP.

Think this way – sending order and getting item gives you 2 HTTP calls. It’s not much but with more services interested in this topic it will only grow. And all of it will be done during main order request. So user facing system will be slowed down by every new service attached. Even if response time is about 50ms with only 4 of them it adds 200ms on top of all other processing.

When your microservices communicates with each other with events you’re getting at least three advantages I can think of right now:

  1. No HTTP, less network connections means less time wasted on the wire
  2. Events are asynchronous so if your critical path is finished you can inform end user about it without waiting for all the rest to finish
  3. If one of services in chain fails events will stay in the infrastructure. You’re not loosing any data.  Rest of system is working as normal. Error is atomic to one place. And after it’s fixed you can start processing messages from where it stopped.

Last point is illustrated in my example. When you request non existent item ItemService will fail. Fix collection adding new index and rerun it. Boom. Order gets processed like nothing happened. Just with a bit of delay.

Conclusion

If you can use this pattern it will give you a lot of freedom. Communicating with event bus of any sort will save you a lot of time and money. One thing to do is just to convince your team and company it’s the right thing to do. But it’s topic for another post ;)

Book review: The 21 Irrefutable Laws of Leadership

21irrefutablelawsofleadershipSo I’ve decided to get more in touch with my soft skills. First book I’ve got is The 21 Irrefutable Laws of Leadership. It’s truly great one.

Book

I was quite surprised that book was so easy to read. Every chapter takes less than hour. Each law has story related to it which easily explains how it works in real leadership. It makes it fun to read. Overall great value and if you’re new to leading it will give you quite a lot things to think about and probably change in your life for greater results :)

At the end of the book you can find all the laws with quotes and references to other books for further investigation. It’s nice as this post supposed to be this kind of thing and by second scroll through book I’ve found this appendix with cool stuff.

To keep my online memo you can find list of all leadership laws below.

The Laws of Leadership

  1. The law of the lid
  2. The law of influence
  3. The law of process
  4. The law of navigation
  5. The law of addition
  6. The law of solid ground
  7. The law of respect
  8. The law of intuition
  9. The law of magnetism
  10. The law of connection
  11. The law of inner circle
  12. The law of empowerment
  13. The law of the picture
  14. The law of buy-in
  15. The law of the victory
  16. The law of big Mo[mentum]
  17. The law of priorities
  18. The law of sacrifice
  19. The law of timing
  20. The law of explosive growth
  21. The law of the legacy

From PHP to Java #2 – Development speed

In my previous article I was hyped about getting insane throughput in my first app. Today I started another app and development speed, using quite difficult architecture, surprised me again.

Spring Boot

As you could expect after previous entry I’m focusing on Spring Boot as it delivers everything more or less out of the box so I don’t need to worry that much about additional tools. It’s also similar-ish to Symfony2/3 so I feel close to home.

Building new app today

Last Thursday I visited London’s Java Meetup which was about CQRS applications, event sourcing and how to use it in microservices. Great talk which motivated me to try to build Event Sourced app.

There is nothing better for the weekend than building application in language and environment you don’t really know what well using difficult technology you’ve never used before. Should give me weekend or two of fun playing with it.

Sadly my project, Java and tools said

no

what’s actually good thing.

Spring & Axon Framework

It’s now 14:23 and I’m basically done with fully event sourced API part with enough materializers to create all views needed for the MVP. It’s kinda crazy for me.

Configuration of Axon Framework (CQRS framework for Java) is very easy. Few services registered in container … I mean Beans registered in Application Context and it’s done. Domain objects working as Command Handlers, listening to events and JPA (like Doctrine) read layer based on MySQL. Everything worked on first run and as it suppose to.

Event sourcing has great part in development speed and I don’t need to care about database at all and I don’t even think about it before I finish all domain processes. After that there is time to reflect what I want to show to the user. What views I’ll have in the application. And how I want to put those data so they will be easy to read.

Amazing. Again.

Conclusion

I’d really like to find some downsides, things I won’t like in Java. I know about Websphere and old “Application Servers” ways and that they are terrible but it’s past. Currently I feel like Java is taking back ground after years of being “slow” and “too corporate”.

I remember doing research years ago, when I was looking for language to learn after seeing no future in Delphi. I’ve chosen PHP as it was growing and for being nice and fast to develop websites. Java felt “too big for task”. Today it seems to be other way around.

From PHP to Java #1 – First production-ish application

So I’ve got a task to check difference in performance between Silex app, Symfony 3 with all goodies and JAVA/Spring Boot. Results surprised even me.

TLDR; because there will be separate post about that – PHP is obviously slower than Java and with proper use of opcache there is no difference between Symfony 3 with JMS Serializer and Doctrine and 60 lines unmaintainable app in Silex.

But it should be about Java. Stack Driven Development with Spring Boot took me maybe 1 or 2 hours in comparison about 1h with Symfony. As I’m still newbie I’m not optimising what so eves as I have just slight idea what I’m doing. App is working exactly the same way. Let’s do testing then.

Environment

I’m having all the apps on my poor Vagrant VM with not so much processing power and I’m using DB which is somewhere in the infrastructure. No cache, no in-memory storage. Everything is as dumb as possible ;)

For testing I’m suing Gatling.io. Great tool with Scala based DSL for test scripts.

We’re set up so let’s do some testing. Goal is to kill all the apps and check what’s the most they can handle.

The test

So PHP died before 50 req/s. It was so disappointing I haven’t checked exact number. Note I’ve done all optimisation on opcache and fpm normal user would do. Nothing crazy tho.

Let’s do Java then. same 50 req/s. No sweat. graph flat at response time about 10ms. Boring. As naive I was pushing by 25 on iteration of test. It took some time but finally got some struggle after reaching 250 req/s. At this point I’m shocked and confused. Mainly because even if I got longer response time I’ve got all the response data when PHP started to throw 500’s and 504’s in my face.

Next step my friend from the office came over and we’ve done some system optimisations on my VM. Then I’ve just

(╯°□°)╯︵ ┻━┻

Last successful test we’ve done got us average 360 req/s. On my VM, on small and slow-ish Mac Mini. And no errors. after that point, when I bumped test to over 400 req/s I simply killed TCP driver on my mac, not application. T bloody CP connections couldn’t be made on that rate on my host. App was fine.

Conclusion

I don’t want to say PHP is bad. It’s still good for frontend and not that high load. But if you really want performance you shouldn’t waste time on PHP IMO. Spring Boot feels like Symfony, all the good tooling is basically done for you.

You’re gaining like 7-8 times better performance out of the box with rock solid stability (I haven’t got even one 5xx from Java app).

Well, choice is yours if you gonna like it.

Why should you use CommandBus

CommandBus and CQ(R)S are gaining a lot of popularity over last year. Today I got question about benefits of the switch. As I used it by default for last year I’ve never thought about it. I’ll try summarize it in this article and in next one I’ll explain how you can change your architecture in life project in reasonable and production safe way. Let’s get to it!

Why should you use CommandBus

CQS stands for Command Query Separation. Simple as it is – every operation which is not just read in encapsulated in Command and handled by CommandHandler. The biggest benefit of that approach is that you have each business operation in separate class. Thanks to that when change has to be made it’s very clear where it should be done and when new feature comes in it won’t interfere with what you already have.

Second argument is very close with first and it’s about communication between dev team and business. Each handler is one specific case and it’s name reflects business language more than what dev team thinks it is. You’re dropping meaningless .*Service classes which grows out of reasonable size very fast.

Stopping unreasonable growth is part of Solid. Single responsibility principle says that each class should have specific functionality. Keeping separate handlers is the best way for that.

Very important in all of that is separation between input and business logic. All you can do outside CommandBus is to create and validate command and send it through with bus. Your controllers are getting very thin and probably you can do half of the things automatically even before it. You also stopping to care where and how command is created. You can execute the same operation from many places without any modifications in your business code.

Cherry on top when you’re creating any API is that you can deserialize your input whatever it is (JSON/XML) straight to object of the command. It saves a lot of time, especially with Symfony 2 where you can create very neat ParamConverter to do the thing for you and throw 400 if validation fails.

Example:

Let’s take UserService from thin controller fat service architecture. You have there two methods – registerUser and updateUser. First one creates user and sends confirmation email. Second just updates fields in User.

First thing broken is SOLID as two operations are handled by the same class. You also have to have dependency on some mailer to be able to send email during registration. But do you need to instantiate it when updating user? No. So we can assume you’re looting time and resources on every update of the user.

Question also is – WTF is “updateUser” – which business process is it? When user changes email? When user changes password? When user is promoted to different role?

Usually all from above processes will use this method and you will have lovely stack of ifs to determine when you have to send confirmation of new email, encode password or just update column with role value. Or even worse you’ll have event listeners on specific fields hidden somewhere deep in code and first new person will add another one because of no knowledge of old ones and after another month you will be adding parameters to enforce listeners to be fired in proper order.

Been there done that ;)

With each process in different class you’re avoiding all of that. You’re doing your tasks with speed and precision saving your time and business money.

From PHP to Java #0 – Beginnings are hard

As I’m slowly working my way up from PHP to Java. I think I can write few words about my experience on that topic.

For now I know basics of basics of the language. It’s not that different and more or less the same with different markup in every language. You can’t reinvent loops of variables :P

Difficulty starts now when I want to write simple app in good way. Currently i’m using Symfony 2, Doctrine, Tactician Command Bus and Twig. Switching to do the same thing in Java means I need to choose similar libraries from java world. Another reason I have to use them is market. To get a job you have to know your tools. You have to know the best tools. And you have to know them well.

Now I’m in the stage when “I have no idea what I’m doing” but after research I decided what will be my starting point.

Web framework – Spring Boot – simplified version of Spring. No XML in favour of annotations and code configuration. From what I see it had good feedback from community and it’s similar enough to Symfony 2 to let me get some results in reasonable time.

Database – Hibernate – The heaviest ORM in Java, probably. But it was a source of inspiration for Doctrine 2 and few years back I was working with it’s .NET clone so I hope I will have only problems with configuration and markup.

Command Bus – Axon Framework – Here is the tricky part. Axon is a huge step from DDD light where state is saved on every request to Event Sourcing. Maybe it’s possible to simplify but I’m not sure at this point. Also Google and Stack told me it’s production proven by banks etc so it can be good direction even when learning curve will be very steep.

Templating – Thymeleaf – Default for Spring Boot. Not bad XML-like markup. For what i need it’s Googleable to extend templates and create reasonable DRY structure of template files.

It’s a lot for a first step into new technology. I spent last month-ish trying to figure out if I can make it in smaller chunks but I can’t see a way where I’m not doing dummy “hello Hibernate” examples.

Plan is to write clone of my ftrack-ddd app on the stack I mentioned in this post. As much as it will be possible I’ll be writing posts about my progress.

Working for free is always a bad idea

As you might know (or not) besides of Software Development I’m also passionate about photography. You can check my Instagram and website if you want. But this post is about my experience in “free freelancing” in both industries.

In software it was waaaay back when I was back in Poland and I had no experience whatsoever. I’ve done few websites “for portfolio”. It was fine. Scope we discussed was done. Problem starts when free job starts to need to be supported with more work.

As with my experience in both industries people will always want more than you’ve agreed on. And it’s not that much about making things for free as making it without “paper” which says what is the scope of contract or agreement.

What I see now is that expectations if not defined are always way bigger than what was told at the beginning. And you, as person doing the job, will always be the bad guy who is doing bad job not delivering.

And I’m not saying it’s good to not deliver. It’s good to deliver more than expected. For sake of having a happy client it’s needed. But when you don’t have specified what really is “the job” you can’t say you’ve done what you were asked for. And trust me, even when you all you were asked for there will be more.

And I know I’m maybe bitching about stuff but tbh I have only 24 hours a day and IMO when I want and I’m spending time doing something for you I’m expecting at least gratitude that I’m doing it and some understanding when we don’t have strict rule on either side and any of us can be caught in more or less relevant to “the job” stuff which can make delays in delivery.

This is what happened recently and as I follow the rule of failing fast my solution is to do things always with written rules. It can be for free if l like it. But with written delivery size and time. Just to avoid unnecessary misunderstandings.