portrait picture

TIMO ZIMMERMANN

balancing software engineering & infosec

Thoughts on Apples Special Event

posted on Thursday 12th of September 2019 in ,

Another year, another iPhone. And a watch. And an iPad. Shockingly no one was surprised. The name change could be expected, but I think there was a 50:50 chance who was right.

There were no design changes beside another added camera. Which seems to be the whole selling point. Faster chip? Sure. Better display? Okay. Another camera? Of course, your videos can now suck in 4k 60FPS and your photos still show a persons head cut off at the top. All of the features make sense and are a technical evolution from the previous generation, but to me they are pretty meaningless.

I do not really watch videos on my phone. I suck at taking pictures. I do not particularly like green as a colour. I do not game on my phone. So upgrading from my iPhone X would basically mean a new battery and zero improvement in any area I care about.

The Apple Watch is one of my favourite devices right now. And if you are not on the 4th generation already the new one surely is a great upgrade. But other than that – I would likely still go for aluminium since I usually wear it while working out and prefer the lightness and I am okay if the watch only turns on when I look at it.

Games and TV+ are the same to me. People celebrate that it is only $5. I believe that a higher price would have killed both products before they even launched. There are many indie developers who will get exposure – not that it pays the bills, but it is something, right? *sigh* – and there are likely some productions which might be worth watching. But before larger publishers and media companies are joining the game, I do not think a price that would roughly equal Netflix would work. I am really looking forward to see how content will increase and develop though.

Overall there are solid improvements for all products and the newly introduced services has potential to become relevant. To me the most exciting thing was Tim not telling us how well everyone is doing but directly starting with product introductions. ¯\_(ツ)_/¯


Security 101: documentation – policies in disguise

posted on Saturday 7th of September 2019 in ,

There are a few things you will hear every start up chant about as if it was an absolute truth. The most prominent one is likely „move fast“ — meaning „all the talk about best practices etc does not apply to us; we have to ‚move fast‘ to make it“. While this might be true in a lot of cases, I have often seen it being a disadvantage, even in the short term.

It is likely that you can get away with a lot of short cuts. Code you write today is probably obsolete in a few months, maybe even in a few weeks. Deciding to cut corners when developing your asynchronous workers likely won’t matter until you have a lot more customers than you currently have. Having a configuration file checked into git, forcing you to redeploy on each config change, can be feasible for some time. Obviously all of the technical debt comes with a few exceptions though, like having your workers separated properly in your code base and not checking in secrets.

What I would advocate against most of the time is skipping documentation. It might be tempting to follow the narrative that documentation is just a burden to maintain and that it will be outdated in a few weeks, but this is purely an organisational problem. If you document features and processes as you build them it will likely only be a very minor point on your todo list when working on large changes. And maintaining documentation as part of your feature work means it will not be outdated (or simply wrong).

This also does not mean you should write prose for everything you do. Changing the way a button looks? I would not expect any documentation at all. Adding a circuit breaker when your email service of choice is down? A sentence in a change log should be sufficient. Developing a circuit breaker solution in house? A few lines as spec to have some guidelines around design decisions are fine.

There is also the sort of documentation you write once and then leave it unchanged for months, maybe years. How do you handle your code review process? Opening a merge request and two people have to approve a change? Maybe one? Is code merged into a staging branch first or directly into production? Just write it down. It is, again, a few sentences and you are done. While it might not be a one time investment, it is a relatively small part of the work you will be doing. And it likely will not delay your product roadmap significantly.

But what do you get in exchange for documenting features, changes and processes? In the end it is still time you are not working on shipping your actual product, right?

First of all: As you scale and grow your team you make onboarding a lot easier. You have a set of documents and resources you can point new employees to to get a basic understanding of how things are working. More senior engineers have a way to see the reasoning behind certain features and design decisions, which, as you grow, will potentially look less optimal when approaching a problem from a best practice point of view. One part that makes hiring expensive is not only sourcing and interviewing, but also onboarding. Being able to reduce your onboarding cost with small, continuous investments will pay off as you scale. Especially when you are forced to grow fast.

The other reason (and this is why it is part of the „security 101“ series!) is that most documentation can serve as policy.

I can already see my infosec colleagues with audit experience reaching for their torches, especially if they were recently going through one and got remediation plans containing „date format on policy X does not match the standard format“. But hear me out, there is actually some truth hidden in this statement.

Startups usually do not think a lot (if at all) about audits, certifications or external controls if they are not operating in a highly regulated environment like finance or healthcare. The day one of those things is required it is usually a panicky “all hands on deck”-situation that no one will have a good time with.

The most common scenario in which this might come up is a partnership with a larger enterprise or selling to another business. Companies will have something like an “vendor acquisition policy” which defines which standards a potential partner or vendor has to fulfil for the deal to go through. Many list SOC in one form or another or ISO27001 for example. The moment one or more certifications is on the requirements list to close a deal you can be sure you will either have to get the certification or the deal is off. There are only a few very rare cases I have seen when someone on the highest level of management whitelisted the deal despite unmet technical acquisition criteria.

Realistically speaking you will not even be close to fulfilling any of those certification requirements by simply documenting what you do, how you do it and what is happening to your system. But you will have a lot of the foundational work done – you do not have to do it last minute. You can, as larger deals sneak up, start looking at the certification requirements and see which processes might need some adjustments, were you are currently lacking documentation or processes and how much work it will be to get your system and company into a compliant state. Having an actionable list and work for which the effort can actually be estimated will be a huge help, especially when you can use it as base for communication with your potential partner to see if you can move forward while working on the last few things – believe it or not, this actually worked in the past.

Some certifications actually make sense. Some not so much. But nearly all have one purpose beside being a checklist item: They help instil trust. Selling to end user and consumers, they show that you actually care. They show that you want to be on the front-page of TechCrunch for shipping a new feature or closing a funding round, not because you leaked PII. While this is not the purpose of many certifications and while you can be fully compliant with a few and still mess up badly, they will instil trust.

Especially for end users and consumers it often does not matter what you actually do, but if they feel like they can trust you with their data. Many will not even know the certification you mention on your marketing page, but they will still feel better that you got one. This might sound wrong to some and be assured, so it does to me. This is not about logic or facts, this is about psychology, marketing and sales. Here, trust is a currency you cannot ignore.

Documenting your system and processes will help you on many different occasions. It is some additional work you have to do, but if you start early and make it part of your regular process it will only be a very small portion of the over all work. So better start today than on the day you need it. Future you will thank you. If you are unsure how and what to document, do not worry, I will cover that in depth in future posts.

If you want to follow this article series you can either subscribe to the general RSS feed or to the tag specific one if you only care about startup security posts.


30 years of Fetch

posted on Thursday 5th of September 2019 in

Over the last two decades I used a lot of software. And I really mean “a lot”. I have seen projects come and go – sometimes the market was not ready, sometimes the developer messed up, sometimes a new app simply was better at solving the same problem, sometimes free software just became, well, good enough. Considering how short lived software is makes the fact that Fetch is celebrating their 30th anniversary even more noteworthy.

I think Jim puts it very well –

Fetch’s longevity has been a continual surprise to me. Most application software has the life expectancy of a field mouse. Of the thousands of other Mac apps on the market on September 1, 1989 I can only think of four (Panorama, Word, Excel and Photoshop) that are still sold today. Fetch 1.0 was released into a world with leaded gasoline and a Berlin Wall; DVD players and Windows 95 were still in the future. The Fetch icon is a dog with a floppy disc in its mouth; at this point it might as well be a stone tablet.

Jim Matthews

I have seen software being celebrated as a big innovation that will revolutionise everything which did not even make the 30 day mark.

I hope one day I will also be able to look back over decades and see software I am or was working on grow, have an impact on peoples life, still be in good shape and have some users who cherish it. Kudos Jim and team!


Automattic & Tumblr

posted on Wednesday 21st of August 2019 in

When it was first announced that Automattic will be acquiring Tumblr I had mixed feelings. Tumblr is still alive, but took so many hits that I was not really sure how much ground it still got to stand on. Plans to merge parts of the architecture with WordPress still seem ambitious to me. And being in a position to argue about and justify the “no adult content policy” surely will not be a pleasant job.

The Verge published an exclusive interview with Matt with many quite interesting points.

Their top priority was not trying to maximize the purchase price. There might even be a corporate reason for the purchase price to be lower, for taxes or something. They were really looking for where the best home was going to be.

Matt @ The Verge

This actually surprises me. Large corporations, including Verizon, usually take a quite different approach at deals. But it is a very pleasant surprise. And considering that Automattic and WordPress, despite all its flaws, is an integral part of the open web, I can see it as a very good home for Tumblr.

The fact that the team can join, a slow transition is planned and a potential independence also speaks for a very thoughtful approach we are usually not seeing in this space. So far this seems like an example of how an acquisition should be dealt with.

Yeah. And some people say, “Well, do you need to be in the app store? Just have a web version.” But apps really are it, and I believe Tumblr is one of the top 30 or 40 apps in the social networking category. It’s usually top couple hundred globally. So their app is a big part of how people interact with it.

Matt @ The Verge

This is totally on point. Without an app Tumblr would lose even more traction. The ease of use, the over all user experience and living on the home screen are essential for an app like Tumblr and for the plans Matt outlines.

And while the AppStore might be the reason why the porn ban was introduced, there should be ways around it. I cannot imagine Apple reviews not being able to find porn on Reddit or Twitter – that would be reason enough to doubt their ability to review anything.

I think there’s a lot of overlap in what both do. I would love for them to interoperate. I do believe that, long-term, there’s an opportunity to merge backend technology so that Tumblr is actually powered by WordPress.

Matt @ The Verge

This surely makes sense from a business and engineering perspective, but please: give me an open source Tumblr frontend for my WordPress setup so I can finally get rid of the garbage I currently have to use to write blog posts. I always liked using the Tumblr interface and the WP web interface just got worse with every single iteration.

I’m getting some strong Google Reader vibes from you. Not that you’re going to build an RSS reader. But it’s still lamented that it’s gone; it was the application that brought together an entire ecosystem of blogs. Is that role something you can fill?

The Verge

It will be interesting to see how ads, content control, integration and monetisation will look like in future. Considering Matts past actions and contributions to the open web there is a chance that we will see a potential competitor for social networks or content publishing emerge that might finally act in users interest and work in an ethical way. I would really appreciate it.

As Matt mentions in the interview it is too soon to see where the journey will lead them, but I am cautiously optimistic that we might get some fresh wind in the online community experience.


Thoughts on video on demand, Netflix and the revival of piracy

posted on Saturday 17th of August 2019 in

Netflix did not really hit their subscriber goal. I actually do not particularly care if the reasons outlined in this article are correct, but I really care about video streaming. And I sadly see the whole thing going downhill so fast that I am waiting for news about „Pirate Bay 2.0 and limewire X“ – our old timers showing up in some new form – bringing back the golden age of piracy.

I am a Netflix subscriber since they expanded to Germany, so somewhen around 2014. 5 years I paid them every single month, the whole family membership, the largest package for the highest quality they can offer. And I am okay with that.

I did not own a TV for nearly a decade and watched all content on my laptop. This worked quite well, until two things happened: Netflix and my wife. So we got a TV. And an AppleTV. All our content on a nice big screen, and I already had a proper speaker system, so this was all setup pretty fast. We started enjoying shows on a 65“ screen instead of a 15“ laptop. Talk about an upgrade. But we still do not have cable or satellite, Netflix and iTunes are our primary sources for movies and TV shows.

My wife sometimes watches the online service of some garbage German TV channel which we are forced to pay for if we want it or not. (It is an astonishingly stupid and people hostile system no one managed to drag through the courts yet…) And last week we bought a movie on Amazon Prime. Not because I like Prime Video – if I would have to pay for it, it would not be available, but hey, it is „free“ – but because iTunes did not have the original dub available.

Recently I started paying for YouTube Premium too, since I am actually watching more YouTube than Netflix and I like some of their shows. Being able to pay instead of being forced to watch ads is exactly the business model people ask for in many online debates around ads, privacy and sustainable business models. I still got a ton of shit for „supporting Google“ and „paying for something I could have for free using $x“. Guess what, this mentality is why we have an ad and privacy-depraved online experience.

For five years all seemed pretty good to me. Then the entertainment industry happened. Everyone wants a piece of the streaming and VOD cake, so Disney started Disney+ for example, with exclusive content. But not only exclusive new content they produce, they also simply remove existing movies and shows from Netflix.

If you look at the larger landscape of upcoming and existing streaming services there is a good chance you will end up in a world where you pay $60-$80 per month, likely even more, to have access to all content.

Obviously people are slightly upset that streaming costs increase by 6-8x per month. We can have a pretty lengthy discussion if the price is actually justified. I subscribe to the theory that the studios and produces can charge whatever they want, if I am willing to pay it is up to me. If I do not consider content worth the price I simply do not consume it. Shows and movies have quite large production costs, so drawing parallels to music does in my opinion not really work out well. In the music industry you also have some alternative income channels like live tours. I mean, I think I would really enjoy seeing Vin Diesel, The Rock and Jason Statham performing a live version of Fast & Furious, but likely „for the lolz“ and not for entertainment value.

But it is not only about cost. With 6-8 streaming sources you will likely be forced to have 6-8 different accounts to manage, 6-8 different applications, one less usable than the other and 6-8 places to search to find the content you are interested in. Apples TV app is trying to streamline this, but as companies still believe in locking users into their shitty experience – hello Twitter – I do not see the TV app being a viable solution in the long run.

With the current developments, there is actually no way forward I see that will serve customers well.

If we have to have multiple subscription based providers a centralized app for consumption would be huge, but is likely not going to happen.

Content being available on all platforms is obviously the best solution, but is likely not going to happen.

A true VOD system where I pay $x to a provider of my choice for the content I want to see, is maybe going to happen. The individual content providers can name their price, but at the same time they do not want a one time payment, but a subscriber.

Remember what happened the last time content was stupidly expensive and inconvenient to access? Piracy. And I can see it happening again considering all the hurdles and money multiple streaming services would bring and cost.

In a small corner of the Internet I can now hear some people scream „ALL CONTENT HAS TO BE FREE“ or „we could pay $10 per month to a central place and they sent $0.01 to content providers which content we consumed“. Take a guess why there are so many shows and movies – because it is a lucrative business. Not going to happen and an absolute stupid idea that is not contributing anything to solving the problem at hand.

At this point I am fairly certain we will actually not be solving the problem, but time will. New platforms with launch, people will subscribe and at some point drop because they do not get enough value for what they pay. At this point prices either will drop or content will be distributed via more channels again. Some platforms like Disney+ have enough money to take a very long time before this happens, but in a few years we will likely only see one or two streaming platforms.

Since this is not a problem I can solve, the only thing left for me to do is figure out what I will be doing in future.

I find myself watching a lot less Netflix lately. New shows which receive a lot of hype simply do not give me a lot. I can watch them, but it is not like I would binge them, a good sign I could live without them. On the other hand I am watching more and more YouTube and Twitch. Especially DIY, gaming and tech channels. To me Netflix is at a point where it is not worth the 15€ per month anymore.

If I really like a show or a movie I usually buy them via iTunes since I will watch them many, many times and I am okay spending a few € for the entertainment value I get out of them. And considering I might have to pay 60-80€ in future, I can buy a few movies and a show per month, not pay anything extra and not hope they are still available when I want to rewatch them.

I do not like the fact that there is DRM in play and that I do not own the media, but that is a trade off I decided to live with for the convenience of having media accessible with one click. Going back to buying things on spinning discs is not really an option for me, there are just too many drawbacks for the impression of ownership.

The entertainment industry is setting themselves up for a stupid battle that will cost them a lot of money and customer satisfaction for some short term profit and a chance at being the winner of the whole streaming and VOD game in a few years. Let us prepare some popcorn and see where this goes, it might be more entertaining than some recent shows.