portrait picture

TIMO ZIMMERMANN

balancing software engineering & infosec

Thoughts on „Surviving Django“

posted on Friday 7th of August 2020 in ,

Daniele, the author of psycopg2, wrote an interesting post about Django and its migration system. Having used Django for well over a decade and having worked on projects either handed to me or greenfield at different scales I cannot agree with his arguments as they are stated.

The migration and ORM system does solve a far larger problem than allowing web apps to switch between databases. Djangos migration system is primarily a developer tool. A rather excellent one I cannot praise enough. It abstracts and standardises the way database migrations are handled. Remember following a document outlining the order of 15 migrations to apply when doing a deploy? Remember trying to add a 16th? I surely prefer running makemigrations. Is it perfect? No. Can a talented DBA get far more out of a database? Absolutely. Is any of that more relevant than safeguards and ease of use for 90% of the apps out there? Definitely not.

Now to the really good part – you can have the best of both worlds! Migrations can run Python and SQL, while still giving you some of the safeguards and making it comfortable to use. They will not get in your way, but spare you from writing your own tooling around migrations or using yet another tool.

Django is a great framework if you use it as it is. Once you start replacing parts you will end up fighting the framework on a regular basis or start re-implementing most of the third party libraries which expect Django to work the way Django does. If you end up at this point you would likely have a better time with Flask and one or two blueprints.


Microblogging

posted on Friday 31st of July 2020 in ,

When micro.blog came around I really liked the idea. Especially not having to post five tweets to express a thought seems very appealing – but that was not all. A platform besides the traditional (are we at the point where the word Twitter and Facebook go well together with traditional?) social media platforms certainly does not hurt. I actually spent more time thinking about various solutions for microblogging and platforms than most would consider reasonable, so of course I also put some thought into micro.blog.

I appreciate what micro.blog did, does and hopefully will be doing for a long time. But there are a few things missing I would consider a requirement before starting to use a new platform. I am certainly open to pay for hosted solutions, in the end when I pay for a service, it is not my problem to keep it online. That is the best part of being a customer!

First of all I am not investing any time in closed source platforms anymore. Micro.blog certainly is as open as it gets, but whenever I would have or want to migrate to my own server I would have to get all the content and somehow build something that can put it on the new server. I cannot just run the app. This being said I still believe micro.blog is one of the best platforms out there when talking about openness.

The most important reason why I decided micro.blog is not the right platform for me is splitting content between my main domain and micro.blog. I can obviously cross-post, link and all of the nice things you would expect to „simply work“ when the platform you are using is not build around the concept of locking a user in. But with every platform some constraints like character limits come into play, forcing me to either cut content short or cross post and have this split where posts live and I simply do not want this.

Having a separate category on my blog for microblogging is easy enough to set up. Coming to this conclusion was easy. The harder decisions came after that.

Should micro posts be part of the regular post „stream“ or completely separated? As in different archive, not part of index and a different RSS feed. Most arguments come down to noise vs focus of my primary post stream with long form articles. Considering that post frequency will still be relatively small compared to Twitter for example I am okay to mix long form articles and micro posts for now.

How much effort should I spend on a post? When I am currently writing an article I usually have my wife read it to make sure I did not do too many horrible things to the English language. If I am writing a longer, more technical article I sometimes ask a friend to do the same, kind of a second review round, with the difference that they are native English speakers. Which is also a very good reminder to spend more time improving my language skills. For micro posts I am more thinking of something like the scene in Social Network where the actor playing Zuckerberg codes Facemash while blogging on LiveJournal. Just without being a horrible, sexist, drunk human being.

Should I integrate pingbacks, webmentions and all the other fancy indieweb features? As someone who removed comments from his blog for a very good reason I am a bit hesitant to invest the time to make it work. But I certainly see the appeal and value in having webmentions set up. My curiosity might win, I have no idea how hard it is to build a small service that plays nicely with a static site generator. Especially considering all the spam pingbacks I am seeing on WordPress today.

After publishing my last post I got more and more dissatisfied with WordPress. This was likely the biggest reason why I did not start microblogging earlier. But with a new solution in sight, it is time to get started. I am actually curious to see how – and if – microblogging changes the way I publish content. Especially if I move more content off of other platforms and just link back or if using other services will still be more convenient.


Just do it – on side projects and the paradox of choice

posted on Monday 27th of July 2020 in

On Saturday I was talking to a junior engineer who sometimes asks for advice and this time he had a few questions around a side project he is planning to start. He wants a blog with a portfolio and he wants to build it by himself, instead of using an off-the-shelf solution. His idea is to write a web app from start to finish, something he had not done yet. And while doing some work on a Ruby on Rails app, he is primarily writing C for embedded devices. His problem was that he did not know where to start, how to build it and at some point he sounded like giving up before even starting.

This turned out to be a longer conversation than I expected, but let me give you an idea what we talked about, what his thought process was and where he got stuck planning the project.

It started with picking a language and a framework. He knows enough Ruby and Ruby on Rails to do some tweaks on API endpoints and add a controller. So you might think he will just run rails new and starts hacking away. But he heard that RoR is a bit too old school and not used anymore. And that Ruby has problems scaling. Golang or Node might be better. People also talk a lot about Haskel.

Even if he would know which language to learn, how would he deploy it? Serverless? On AWS or Heroku? Maybe a simple virtual private server. Should he use PostgreSQL or MySQL? CloudFlare or CloudFront? Does he need any deployment script or automation?

We all love to play with new, shiny toys. Sometimes we are sick of them and just use the tools we know, because we can make an educated guess if they are capable of getting the job done. And if we are wrong we usually know how to correct our mistakes. But for someone just getting into this beautiful mess, all they see are blog posts explaining a brand new technology that will change everything and solve all the problems other stacks have. Also – obviously – blog posts explaining why the brand new technology is stupid and you should be using something else. You know the drill.

Real talk – it likely* does not matter.

*we are still talking about side projects not meant to generate any revenue or provide a service to a user base**

In this specific case we are talking about a personal website. What is the worst that can happen when it is down? Some people will not be able to read the content you spend a lot of time on creating. This obviously sucks. But you are not loosing revenue. You are not angering any customers. Downtime sucks, but you likely beat yourself up over it way more than anyone else will. We all had systems go down, we all did stupid at some point, but this does not mean you are any less of a professional, especially if you are building a project which is not production critical in your spare time to learn something new.

So which technology should you be using? The one you know. Spin up a blog in RoR, use SQLite and get the project online. SQLite does well for read heavy workloads. It will be fine. You do not need a CDN. If you are worried about server load put Nginx in front of your app and turn on caching. Latency might be a bit higher, but your RoR blog will most likely even survive unexpected exposure handling a few hundred or thousand requests per second. Deployment? `git pull` is good enough for one or two changes a month.

What is more important than getting every detail right, creating a project that represents the pinnacle of engineering, changing the way we think about blogging, is actually starting to build something and finishing it. Even mostly finishing it, but getting it online is a major step.

You can always iterate. Maybe out of necessity if your blog actually goes down way too often, maybe as a learning experience because you really want to know how to setup a load balancer.

But you will certainly have more fun and learn more doing something stupid that blows up than not doing anything.

Thinking about next steps, your stack and how to get from A to B is obviously a skill you should acquire and improve, but do not let it get in your way of actually doing things. Especially things you most likely take way more serious than anyone else who just enjoys them when they exist and / or work.

In retrospect it is easy to see all the things you did wrong and you would not do again. Usually we call this experience. But to actually be able to look back at something you have to start. So when in doubt listen to Shia LaBeouf.


Security 101: Internal Threats

posted on Tuesday 21st of July 2020 in ,

If you follow tech news you will likely have seen by now that some high profile Twitter accounts were used for a Bitcoin scam. There was a lot speculation what happened before Twitter posted an early update of their investigation. The Times also published an interview with people who seem to be involved. This event raised a lot awareness for internal threats, so it seems like a good time to address them in the Security 101 series.

Over the last ten years I have not worked with one startup that was actually aware of internal threats being a thing. I mentioned them a few times in past articles in this article series as they are not only misunderstood – or simply not acknowledged at all – but also extremely hard to guard against. The really tricky part is that they are mostly irrelevant for you until they suddenly become relevant and turn into a big deal.

Before jumping into more technical details and steps you can take to prevent, mitigate or at least detect problems origination from within your organisation let us talk about the two different problems we will cover:

1.) Social engineering and phishing. If you thought only your users might fall for those two you are in for a surprise. There is a reason why Google moved from TOTP based two factor authentication – and did not even consider SMS an options as the risk for SIM swap attacks is not negligible – to security keys and seems to have eliminated phishing as an attack vector. People will sometimes make mistakes and once you have an account it becomes easier to gain additional access.

2.) Bribes. This does not only happen in movies. The more interesting as a target you are and the more lucrative it seems to breach your system the easier it will be for an attacker to spend money for some form of access. No matter if it is one of your customer support representatives who already was planning to leave your company  and sees an easy way to get an additional year of salary this way or your catering team snatching a laptop off a desk. Once an attacker is able and willing to spend serious money things become really tricky.

Both of threats are usually not relevant to many companies until they are. You might hit hockey stick growth overnight and suddenly your company is worth three social networks or you might just have created an innovation which might disrupt a large market. You start to grow rapidly, onboard new people and suddenly those two attack vectors exist in an amplified state. When your company only consists of two technical founders they are already present, but the chances for them to become a problem are hopefully significantly smaller.

As previously mentioned, using a security key will bring you a long way. They are relatively inexpensive, there are tutorials and libraries to integrate them with most stacks and they work on desktops, laptops and mobile. Eliminating phishing – or at least reducing chances significantly – will be a huge step in the right direction. But as we already established this is only one part of the problem we are facing.

As you grow one of your most valuable tools beside common sense will be a solution for Security Information and Event Management – aka SIEM. But it will take some time to get there. And you will need an experienced team who actually knows how to configure it properly. Luckily we can get some of the benefits with a lot less work and less expensive systems.

At its core you want two things: collect all your logs in one place and detect if something looks wrong. You obviously want a notification if you detect something.

Defining what „wrong“ means in this context is hard. Especially when starting out and not having a good data set. Or an idea how users actually use your product. Worst case you define „wrong“ wrongly – sorry, I had to get this one in – and end up throwing alarms for legit events every five minutes. This will most likely lead to people ignoring alarms after a day and all the value you can get out of your system and later on a full blown SIEM is gone. Or you simply do not detect anything. So all you do is burning CPU cycles running a system without value. SIEMs – even systems only providing part of the functionality – are hard.

To give you an idea for the incident we have seen with Twitter:

In hindsight those rules are obvious, but how many of those would you have setup without the incident? The first two are relatively hard to fine-tune. How many are „multiple“? What is „the same message“? What happens at New Year’s Eve?

The last one on the other hand is something you should always setup. Even if you hardcode the rule in your app. If anyone touches sensitive customer data which can lead to an account takeover, send a message to everyone who might be on call or engineering.

It would be even better if you do not make this feature accessible in a straightforward way. Make people jump through hoops like two people confirming the removal of two factor authentication. You obviously add work and service cost, but you also make exploiting this functionality which might actually be used for customer support way harder.

On the technical side I already mentioned hard coding the most important alarms into your system. This works well until engineering is compromised. But when you get to this point you have to worry about so many things that it is nearly game over for a regular startup. A little bit more sophisticated would be writing all your logs to a managed database or data store like S3. A small server can parse those logs checking for pre-defined events and sending you an email or a Slack message. While this is not highly available, might blow up from time to time and is quite a bit additional work, it is a solid foundation to build on. What I have seen most startups do is pay a lot money for a SIEM without accomplishing anything beyond what the basic alerting system we talked about is capable of. This is understandable as they lack time to get familiar with a tool complex tool in a domain they do not know.

As with many things we are discussing here, it is just a start. It will not be the last time you have to invest time and money in this part of your system. The more customers you have, the more data you store, the more relevant your application or platform becomes, the more important logging and alerting will become.

One of the most unpredictable threats you will face will be of internal nature. Your employees, coworkers and consultants and anyone entering your premise will become a potential attack vector. And as unpredictable is not enough, most of your mitigation strategies will result in you making everyone’s life harder. As you can imagine you will not be their favourite person when you roll out required changes. But this is sometimes the price to pay to ensure to keep your company and data safe and secure.


Thoughts on WWDC2020

posted on Thursday 2nd of July 2020 in ,

Another year, another WWDC. I could be wrong but this felt like the most consumer focused WWDC I have ever watched. And it is one of the few which actually will have an impact for years to come – think PowerPC to Intel or the introduction of the iPhone. I did not immediately want to post thoughts on all the announcements as I wanted to play with Big Sur first. While it took a bit more work than I would have liked, I got a virtual machine set up and had a chance to spend some time with it.

Big Sur

There is a lot going on, but the biggest change is likely the UI. It is closer to iOS and iPadOS than anything we have seen before. Somehow Apple managed to still make it feel like a Mac, not a scaled up iPadOS version. If you like the design is up to you, tastes are different – personally I am looking forward to upgrade my main system later this year.

The animations are subtle, but I can see people already complaining that actions now take 0.5 seconds longer than before. After all those years – and hearing the same complaints about Windows – I still did not figure out what people do with the 2 minutes a day they gain on not waiting for animations to finish. But I am sure it is life changing.

To me the biggest change is Notes. It finally does not offend the eye anymore! I cannot express how much I hated this slightly yellow, textured design. I am glad it is gone. Reminders also received some nice updates. While it will not compete with Todoist, it might be enough for some people to not buy the next upgrade of Things.

iOS

Apple really tried getting rid of things taking over the whole screen, like incoming phone calls or Siri. But this is mostly a visual change: While Siri is active you still cannot interact with the elements on screen you are seeing. And this seems intentional – in an interview Greg Federighi mentioned they tried a version where you could interact with the system and it did not feel right.

The feature that struck me as odd one is picture in picture. I get it on the iPad 12.9”. While I am not a big fan, it is okay to use from time to time. But phones have limited screen real estate. I cannot imagine actually looking at a video in PiP mode on any phone in existence right now.

(So here is a bit of speculation that might be too forward thinking – what if… Apple works on a foldable phone? Maybe two separate screens with a hinge design. Suddenly there is a lot more screen. And taking over half of the screen available for a phone call or Siri while still presenting information and working on the other screen makes the design decision more coherent.)

Apple Clips could be a game changer. There is a lot of potential – to succeed or fail – and Apple likely has a better chance getting people to adopt it than Android had when they shipped their version. What I really dislike is that it is not an open standard. For businesses to support something like App Clips would be a lot easier to justify it they would not need a customer study first to see if the majority of their customers actually have an Apple device.

(I will not go deep on the whole widget and icon organisation topic – I have used this on my Android dev device enough to know that I literally do not care.)

iPadOS

Search is likely the big one – especially with the magic keyboard it feels more natural to have an macOS like Spotlight experience than the thing we got right now. It also looks like a visual upgrade to me and it makes multitasking feel a bit more natural.

The really big one for me here is – again – Notes. While the handwriting recognition is nice (if it works), the fact that it finally has shape recognition means I can do most of my sketching for architecture diagrams in Notes. Together with the initial draft of the document, primarily because Notes on macOS does not look horrible anymore.

Apple Silicon

I am waiting for an ad screaming “don’t call it CPU”. I do not think I have heard the word silicon so often in a 20 minute video as in this keynote. It is a change people predicted for a very long time and now we will finally see systems ship with Apples own Silicon.

I can easily see this being amazing for notebooks. We will see how the first iteration performs, but considering what most people do with their laptops it will be good enough. Likely not for many engineers, media professionals,… but regular users writing emails, documents and using the web. Considering my iPad Pro has a 36.71-watt-hour battery which lasts for 10 hours imagine how long a laptop would run if the battery only stays the same as in todays models. We are talking truly all day battery life.

Considering they still plan to announce Intel based system as well and that they surely are a few years away from being able to replace a Mac Pro for example, I do not see them deprecating current systems soon(tm). This does put a lot more pressure on the compatibility layer. No one likes “you cannot run this software on your system because reasons you do not understand” – again, we are talking about regular users, not software engineers that know the difference Silicon can make.

While I do not see them merging macOS and iPadOS / iOS, I am curious if portability of apps means I will finally get Xcode on my iPad. What I would really like (and I have been asking for that for years) is to be able to plug a monitor into my iPhone or iPad, connecting a keyboard and mouse and switching the UI to macOS. There is a very good reason why Apple will not do this though: They would cannibalise their low end laptop market. There is also a very good reason why Apple will do this: They can streamline their product lineup and upsell tablet upgrades. Time will tell which way that will go.

The rest

Nothing in watchOS 7 actually excites me. Literally nothing. I also do not see a lot they might want to change at this point, I think we will first see a new revision or new type of wearable which might justify significant OS changes. In the meantime some actually useful standalone apps would be nice.

tvOS – I am sure someone somewhere will use AirPod sharing once. YouTube in 4k is very much appreciated.

Automatically switching between devices will either be an amazing experience or super frustrating when your video call drops audio because you receive a notification on your phone. As I am not running any of the betas on real devices this one is hard to test.

Virtual keynote

Honestly, I like the style. A lot more information than you usually see in a keynote and a faster pace. It felt like the right mix of acknowledging the situation and still being light hearted. And whoever worked on the Apple Silicon intro – kudos, usually I would expect something like this to end in a cringe fest, which it didn’t.

I am sure Apple will move back to a physical event as soon as possible. But I hope they keep the current way they engage with developers and share content. It feels a lot nicer than the years before and the iPad app is actually pretty good.