<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ian Nowland on Medium]]></title>
        <description><![CDATA[Stories by Ian Nowland on Medium]]></description>
        <link>https://medium.com/@inowland?source=rss-81db999c97a7------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*MES4CzKyZQIce9K7yeYUDg.jpeg</url>
            <title>Stories by Ian Nowland on Medium</title>
            <link>https://medium.com/@inowland?source=rss-81db999c97a7------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 02:10:21 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@inowland/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com" rel="hub"/>
        <item>
            <title><![CDATA[Configuration Should Be Treated Like Code]]></title>
            <link>https://inowland.medium.com/configuration-should-be-treated-like-code-c7167dd25b7c?source=rss-81db999c97a7------2</link>
            <guid isPermaLink="false">https://medium.com/p/c7167dd25b7c</guid>
            <dc:creator><![CDATA[Ian Nowland]]></dc:creator>
            <pubDate>Sun, 09 Oct 2022 03:41:31 GMT</pubDate>
            <atom:updated>2022-10-13T00:11:55.353Z</atom:updated>
            <content:encoded><![CDATA[<h3>Amazon.com 2007 — My introduction to “servware”</h3><p>“Configuration is Code” was a rant I wrote in late 2007 after I had been at Amazon 18 months. Before that I had been doing embedded development — where everything you shipped you had to accept it could never be changed. Whereas Amazon was my first experience in a “you build it, you own it, you run it” world which opened up a lot of flexibility for iteration on both product and system. This is a no-brainer thing for all today, but this old Steve Yegge <a href="https://sites.google.com/site/steveyegge2/its-not-software">“its not software” essay captures how revolutionary it was</a>. He used the term “Servware” which never caught on, but which no real term for the artifact ever caught on.</p><p>So looking at configuration in “Servware”. What I noticed was that my new teammates would spend a lot of time intensely arguing about the right abstractions for code, and the right way to architect data structures and systems for reliability and scale, then totally lowered their bar when it came to configuration — the magic numbers and strings that affected system behavior. Here they just “no brain” accepted a bunch of things they never accepted of code:</p><ul><li>A lot of copy and paste in definition of values (i.e. the YAML hell you see with k8s today)</li><li>Almost no automated testing of system behavior in light of likely values</li><li>In some cases, it was in a system with no version control</li><li>In all cases, extremely fast rollouts of changes to production, bypassing all safety checks that the new system is running on some nodes before deploying to more nodes.</li></ul><p>From new eyes to the “Servware” world, my teammates seemed to be unnecessarily carrying across a “software” world configuration file/registry mindset. My argument was that since we were operating all instances of the developed system and could deploy at will, “configuration” was an outdated notion. If we needed a new configuration value in production, we could change it in the codebase, run it through tests, and build and deploy in minutes. Thus we should move all the definitions of configuration values to a type safe, Turing complete language. Since we were a Java shop, that meant treating it all as Java code (i.e. rather than a Java constants file).</p><p>This was influential within my organization, which was teams of generally ~100 hosts worldwide, well served by Amazon’s build (Brazil) and deployment system (Apollo) — i.e. you could go from checking in code to a canary tested full fleet deployment in ~15 minutes, staggered across data centers. I then joined AWS EMR in 2008 with a similar setup and had to drop the type safety for some of our Ruby code, but once again we saw good outcomes in treating arbitrary constant values the same way we did arbitrary logic constructs.</p><p>So all simple, right — configuration is code? This idealist mindset was challenged when I moved to EC2 in 2010.</p><h3>EC2 2010 — Configuration in the world of Servware Infrastructure</h3><p>One thing to capture up front here as in 2022 people have a hard time believing it, is exactly how poor quality EC2 software was not only by modern standards; but even by Amazon.com servware standards of the time. This was in the period where Andy Jassy would give AWS projects a codename with a number which was the number of days you were expected to launch in; and the numbers were less than 200. In hindsight it was likely rational; a land grab to determine what were the right building blocks for a massive software infrastructure business. But from an engineering perspective, it encouraged/empowered a “we don’t have time to think about tomorrow” mindset in engineers; which over EC2’s 4 years of rapid growth as an infrastructure business had built up a massive pile of hard-to-change tech debt. At the point I joined, it was widely known internally as a place you went to burn out, between getting paged 30 times a week and the difficulty of changing the systems that were doing it. Indeed the year I joined there were _zero_ other internal transfers — my reason for joining was simple curiosity — “how bad could it be?”</p><p>The answer was “pretty bad” —</p><ul><li>The &gt; 30 week page rate and difficulty of changing systems was real.</li><li>EC2 was already at over 20,000 hypervisor nodes, and would be at over 100,000 in 2 years</li><li>On the hypervisor five different languages were used to write production code (Bash, Ruby, Python, Perl, C++), and they needed to interact. That is, there was no easy code based way to share configuration values across systems -so my whole “pick one Turing complete language for config” idea was bunk.</li><li>On top of this, Xen virtual machine behavior was tied to various system limits you could only program through sysctls, where the way is literally old school config files. Anything trying to work around this with something more “modern” was going to run in to race conditions on the case of a system reboot.</li><li>EC2&#39;s build system was not the Amazon standard, instead it was a bespoke mess which failed to scale to a larger codebase as the EC2 org grew, taking an hour to produce a binary (TLDR was lots of Ruby and some very slow unit tests and no ability to bypass them selectively)</li><li>EC2&#39;s deployment system, to use the word “system” kindly, was a parallel SSH mechanism to lay down RPMs; rate limited by the number of SSH connections a single deployer host could sustain, which was not high. This meant by the time we hit 100,000 hosts, deployments took over an hour at the very fastest.</li></ul><p>The worst aspect of this world was a few teams had decided to work around the limitations of the deployment system and build their own ad-hoc configuration deployment mechanisms, all with the same idiom: a cron-based agent fetching config files from S3 which they would apply as soon as the next job runs. These had exactly _zero_ of the controls my “configuration is code” screed talked about. But in the hands of the experts who built them, they were incredibly useful for quickly solving business problems, leading to quick promotions for said experts.</p><p>Like a lot of EC2 at the time, it was a standing but fragile house of cards, set up to start falling apart as soon as the experts who built them started moving on, which between burn out and looking for the next promotion, they did. However, between the inherent challenges of the system, and the tech debt, there was no silver bullet getting to a better place.</p><h3>AWS EC2 2013 — Configuration working Around Idealistic Deployment System</h3><p>Progress was a lot of rewriting of the individual systems, and in doing so thinking hard about what aspects needed to be dynamic (i.e. fleetwide in minutes) with a lot of safety checks that limited flexibility, and which aspects should be static and go through the rom scratch rewritten code deployment system. This turned out to be a 6 year mission — so for instance when we released the c3 instance type in November 2013, when I did a “night before the announcement” configuration deployment to fix how we were doing EBS throttling, I used the old SSH system, as the new system had been written with a “one AZ per day” enforcement baked in, with no flexibility to override for pre-production hosts like in my case.</p><p>The implementation of that deployment system comes to a general challenge, which is in a complicated high entropy world and humans trying to get stuff done, systems that are too ideal in their design often end up causing more damage than pragmatic ones with more obvious flaws. So beyond the one AZ a day thing being baked in, that system had a worse problem that between creator idealism and compliance regulation, it was not written with any type of API to automate deployments; it required a human push a whole bunch of buttons to make the deployment go; and further push more buttons to react to a whole bunch “looks suspicious and triggered monitors” events during the deployment. Given the entropy of a 100,000+ fleet of servers, this meant manually looking at the logs and clicking a “resume” button of O(tens) of hosts every deployment. Combined that then with the “one AZ a day” policy, doing a deployment was about half a week of toil, with the rest of the week with work difficult to focus, waiting for the page so you could unblock things as soon as possible, else the whole thing slips a week and your leadership chain yells at you for slipping a date.</p><p>This was especially painful for one of the EC2 networking teams that managed the physical network firewall rules of the hypervisors — since the network was constantly expanding, they were deploying config changes about once a month. They tried to work with the deployment team to cover the case, but due to the combination of idealism and the difficulty of revisiting idealistic design assumptions in an already complex system, the deployment team was not able to accommodate.</p><p>As a result the EC2 networking decided to build a “better” version of the “fetch a file from S3” mechanism — literally — “but we will build in safeties!” The safeties were exactly two — command line switches that someone had to include as more risk was taken, and a “fall back to last” behavior of any primary consumer of the configuration. Now, I am both a natural idealist and had lived the white knuckle terror of making EC2-wide changes in the earlier S3 syncer systems, so I strongly argued these safeties were inadequate and this was a massive mistake. But I got told the toil was too high, this was the only solution to fix that could fix that soon, and so I needed to do the Amazon “disagree and commit” (i.e. shut up), so I did.</p><p>Play forward about 18 months and I inherited this system. Now I knew it was flawed, but it had been 18 months, I didn’t want to ask an engineer on my team to do the high toil thing (especially since with the pressure removed the deployment team had built nothing in the meantime), and so I normalized the deviance, and gave the system to a strong but junior engineer.</p><p>And in karmic fashion, about 3 months in, he had an emergency change he needed to roll out globally quickly — a product manager dropped the ball on executing network changes so we were going to literally run out of IPs for new instances. So the engineer plugged in the needed command line safety overrides and deployed a change to over 1 million hypervisors worldwide in less than 5 minutes. And while the system he deployed to picked up the change fine as it was “just more config in the list”, it tripped a bug in an adjacent agent running on each and every hypervisor. Luckily this just caused a crash of the agent and when it restarted it could use the new config, and so apart from a very brief blip of some connectivity on the crash, there was no further impact. But given what this other agent controlled, this was just luck; we were very close to a global outage of new network connectivity to every EC2 instance in the world. Obviously the safety was then massively restricted, and we now had impetus to get the deployment system development team to prioritize feature to better suit our needs. But in my memory what remains is the downstream bad outcomes that happens when systems are built too much to an ideal.</p><h3>Takeaways</h3><p>So where I came to for configuration, in the way the values of numbers and strings have a direct affect on system behavior, configuration has little that distinguishes it from code. So it should be treated like code.</p><ul><li>Ideally all configuration can be written in a Turing complete language that minimized copy and paste</li><li>Ideally all configuration is statically checked for type safety</li><li>Ideally all configuration is regression tested</li><li>Ideally all configuration is in a version control system</li><li>Ideally all configuration is deployed to product with the exact same safety checks as non-configuration, through the exact same system</li></ul><p>On top of this though, I repeat “Ideal” in the above as you need to be thoughtful in all cases; I learned the hard way to be wary of the combination of idealist and monolithic approaches to engineering tooling that force all use cases into the presumed ideal usage. They end up causing more harm than something less “pure” would have, as in the real world engineers will just work around a non-functional ideal. In particular the case of configuration, by using a central store as a means of deploying numbers and strings that change system behavior. This is a fast propagation mechanism, but not a safe one. So in your idealism, you have empowered the development of expert’s “quick and dirty” mechanisms that become the massive outage foot guns of the future.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c7167dd25b7c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[7 Areas Software Dev Manager Annual Review Template]]></title>
            <link>https://inowland.medium.com/7-areas-software-dev-manager-annual-review-template-232d27774d66?source=rss-81db999c97a7------2</link>
            <guid isPermaLink="false">https://medium.com/p/232d27774d66</guid>
            <category><![CDATA[engineering-mangement]]></category>
            <dc:creator><![CDATA[Ian Nowland]]></dc:creator>
            <pubDate>Thu, 06 Oct 2022 23:35:42 GMT</pubDate>
            <atom:updated>2022-10-06T23:40:26.103Z</atom:updated>
            <content:encoded><![CDATA[<p>For the last 5 years, I have been using the <a href="https://medium.com/@inowland/the-seven-areas-of-software-management-be89d213ea7">7 areas of software management</a> as a rubric for how I do annual performance reviews of engineering managers, at all levels — from line managers to VPs. With some tweaks over time, I have found it very useful as a framing mechanism to a frank conversation on successes and misses in outcomes they achieve, and mechanisms they could be using to improve things.</p><p>So, following is a template version of that mechanism. One thing to highlight before it is I include a couple of templated areas that I believe are very useful to frame the feedback. One is the data from whatever survey you do across all of engineering, highlighting their relative average performance across each area. I do this as while it is clearly biased, team happiness is one of the true objective areas of performance you can talk about.</p><p>The other though is more controversial, which is I paste in all feedback from peers and directs without names but also without any attempt to anonymize who is saying it, nor any immediate attempt to explain or interpret it. Rather, when I do want to explain it, I do it down below in the seven areas. I do this because:</p><ul><li>I think we all know that attempts to generalize/anonymize feedback greatly lose the impact of detail.</li><li>I think one of the hardest challenges of a leader is maturely listening to feedback even as you strongly disagree with it. I prefer to be open about that challenge.</li></ul><p>To be clear though, I wouldn’t do this for people &lt; 5 years into their career, as they are usually still too attached to seeing all feedback through the lens of their ego and ambition.</p><p>In any case, below is the template, as well as some sample text that shows how I talk about performance in the area.</p><h3>Last Engagement Survey Scores</h3><p><em>&lt;pasted verbatim just for reference&gt;</em></p><h3>Peer Feedback</h3><h4>Strengths</h4><p><em>&lt;pasted verbatim&gt;</em></p><h4>Areas to Improve</h4><p><em>&lt;pasted verbatim&gt;</em></p><h3>Directs Feedback</h3><h4>Strengths</h4><p><em>&lt;pasted verbatim&gt;</em></p><h4>Areas to Improve</h4><p><em>&lt;pasted verbatim&gt;</em></p><h3>Performance Evaluation</h3><p><em>This is a mechanic based on the taxonomy of management areas described </em><a href="https://medium.com/@inowland/the-seven-areas-of-software-management-be89d213ea7"><em>here</em></a><em> . For each, I evaluate with a 10/30/50/10 Top/Exceeds/Achieves/Needs-Improvement scale, under an idealistic (i.e. not calibrated) weighting of my expectations of the level.</em></p><h4>People Hiring/Developing/Culture (Value)</h4><p><strong>Org Hiring/Developing (Value)</strong></p><p><em>This is only for managers of managers. This evaluates team growth, attrition and promotion across your org, where it is good/bad, and your personal involvement in making it better.</em></p><p>Example (Exceeds at Sr Director level): Overall you are doing well here:</p><ul><li>Your team needed to hire quickly at over 50% year over year, and you have ensured they did, spending a large amount of personal time with recruiters, in interviewing and onboarding managers to make it happen.</li><li>Attrition has been negligible.</li><li>While there is some feedback from one of your directs about promotion rate, I know your org is not one that the data shows is slow in that regard, thus I tie it to the next section that you need to be more direct and transparent with your directs about their path and likely timelines to promotion.</li></ul><p><strong>Directs Hiring/Developing (Value)</strong></p><p><em>This evaluates team growth, attrition and promotions across your directs.</em></p><p>Example (Needs improvement at Sr Manager level): Firstly, I want to acknowledge some circumstances that were a bit beyond your control given overload from carryover of past commits and under-hiring from your last manager, and also that you are already addressing changes. Still the feedback from your directs is pretty clear about lack of feedback, you missing 1:1s, and you making decisions for them in a way they feel disempowered. Now again, from our 1:1s I have no doubt that you want to and are in fact already addressing this, so the main thing is to make sure to stay on top of it even if things become hectic again.</p><p><strong>Culture (Value)</strong></p><p><em>This evaluates the data from engagement survey results, and what personal effort you have put into building a good org culture.</em></p><p>Example (Needs Improvement at Director level): it is clear from the engagement survey that your team is significantly less engaged/happy across all areas surveyed. Now as I have talked about, my experience with that case is usually it’s more about overall sentiment than specifics of all areas. And I think in this case, it’s clear in your teams a lot of the overall sentiment problem is a lack of feeling of connection of their work having value to the business. Now, I know some of this dates back to your lack of alignment in team mission with your former manager and so you were forced to take some hires who would never align. However I think we have got that to a good place, so going forward I think you need to engage in more practices so your team understands it and feels good about it. Things I have seen here are quarterly all hands, fortnightly demos/talks sessions, org wide update emails on wins and challenges. I don’t think you need to do them all at once, but I do think in this next period you need to find the time to start doing one.</p><p>Example (Achieves at Sr Director level): On one side your engagement survey results are exceptionally good, and from the feedback you are getting from your directs, I have no reason to think that will stop as your scope grows — you build happy teams with strong managers who have them understanding how they add value to the company. The one thing holding me back from “Exceeds” though is you have not done anything yet to bring your disparate teams together to a common mission. Be it all hands, cross team guilds, etc. I do think this “story of why we exist” is an important part of leadership, especially looking towards VP level.</p><h4>Engineering (Value)</h4><p><em>This evaluates:</em></p><ul><li><em>Do you have good engineering strategy, architecture and implementation?</em></li><li><em>Are you achieving this as much as possible by delegating to senior/staff engineers rather than having yourself (or your managers) do it?</em></li></ul><p>Example (Achieves at Director level): I would say until this is an area you struggled on the architecture side until the last 12 months. Now a lot of that was beyond your control, a lot of tactical asks, not enough people to do them, and so the right thing to do was wedge the asks into the existing systems. However with headcount growth you now have the ability to pivot, and you have personally taken that initiative to push along staff engineer YY’s, new architecture proposal of XX, which there seems to be broad agreement it’s the right thing. So all trending good; the main reason for saying “Achieves” in this period of evaluation is that implementation is only starting, and even in the next period, you have to cleanly hand this off so YY is clearly driving it, as sometimes it comes across too much as your technical vision.</p><p>Example (Exceeds at Sr Manager level): This is a total turn around from last year where the feedback was you were being a player-coach dominating the technical vision of your team. You hired a staff engineer, have promoted some of their ideas even as they conflict with yours, and helped them establish a relationship with your senior engineers where they feel a lot more confident they can own their own designs through a constructive RFC process. So beware of backsliding, but otherwise; well done!</p><h4>Product/Portfolio Strategy (Value)</h4><p><em>This evaluates:</em></p><ul><li><em>Do you have a documented strategy of what you are investing in and why, with time frame appropriate to your level — (Sr) Manager=6 months, Director=1 year, &gt;Sr Director=3 Years.</em></li><li><em>How good is your strategy in terms of all stakeholders (engineers, product management, management peers, eng leadership) being on board with it?</em></li></ul><p>Example (Needs Improvement at Director level): This is the main regression from the prior review, and it is somewhat difficult in that you did not have too many options at the time due to other things taking a lot of your time. However you did have a few options, and more than that, should have a lot more opportunity going forward. So getting to the point: clearly your peer feedback shows they are questioning your team’s decisions on not delivering enough incremental value on current systems, instead focussing too much on new ones. Now, while I personally think you made the right choices given tops down priorities we could not deliver in the older systems, part of Product/Portfolio Strategy is the management of expectations, and it’s clear you did not do so here. Now, going forward with 3 TPMs, a Staff engineer and an extra Eng Manager, and a team finally free of urgent business enablement goals, you has the resources to better communicate your roadmap and its rationale of priorities.</p><p>Example (Top at Director level): Can’t say enough good things here. As we all know, the Product Management org at this company can’t think beyond 3 months in terms of priority, which is fine for many systems, but yours have outgrown that as a planning timeline and you were starting to disappoint peers that you seemed to be moving too slow. I asked you to take the initiative to do what you can, and you not only put together a high confidence roadmap, you got your product parter to collaborate, and the complete buy in from your peers. That includes in the decision to consolidate one peers system XX with your system YY. Finally a couple of peers took your template and have started to use it themselves. Overall, really well done, please continue.</p><h4>Execution (Value)</h4><p><em>This evaluates:</em></p><ul><li><em>How your teams are delivering against goals — both formal ones like OKRs and larger commits, but also informal ones where you tell me a certain thing will take X time and I sign off on you doing it.</em></li><li><em>Otherwise the absence of feedback that people are concerned that your org isn’t delivering the right things, or isn’t delivering in a timely manner.</em></li></ul><p>Example (Needs improvement at Director level): I think we both know this was not a great year for your teams for execution. It is fine to miss dates. It is not fine to miss dates across multiple teams and multiple times with extremely short notice from the date. Yes, some of this is on your TPM, but you need to be active in your engagement with them in the steady state, rather than ignoring things until it comes to a head late then trying to come in and micromanage. You have many things you do naturally well as a manager such as people management. This is not one; so you need to find a way that you are engaged so that myself, my leadership, your teams, and your peers all don’t get these massive last minute surprises.</p><p>Example (Exceeds at Director level): In terms of seriousness to the growing business in owning mission critical high tech debt systems so you need to balance operational excellence but also needing to execute on new initiatives, there are few harder tasks at the company than your org. Yet in the face of things hard tops down dates on initiatives XX and YY, you not only kept your teams executing; but had them as early executors, leading the cause for other teams who thought you would be the long pole. This comes with costs in terms of asking your engineers to do more toil than most, but was absolutely what the company needed in this period, and you led your team to deliver.</p><h4>Operations (Value)</h4><p><em>This evaluates:</em></p><ul><li><em>Are your orgs engineers/systems causing lots of incidents, or a fewer number of really bad incidents?</em></li><li><em>Are your orgs engineers burning out on toil or pager load?</em></li><li><em>Are you spending the appropriate amount of time (not too much or too little) ensuring their org is putting the appropriate amount of time to ensuring so.</em></li></ul><p>Example (Exceeds at Director level): I am a “proof is in the pudding” judge here, and while I do have some concerns about level of tech debt that built up, in terms of actual outcomes to date, your team has done a great job staying on top of it by all metrics, and I know you are doing a bunch of processes each week to stay on top of it. Thus a warning that you may need to find a better balance ahead, but to date you have done a great job of supporting the company’s rapid growth without burning out your team.</p><p>Example (Needs Improvement at Director level): It is clear that your orgs focus on delivery has cost them reliability. Your sev1 incident rate has been high at &gt; 2 a month over the last 6 months, for a wide variety of root causes. It has not cost the company a large customer yet, but it’s clear we are walking the line. I think the question for you is how do we start getting ahead of the game here — your “ I have 2 lifesavers to bring in for all my bad issues” approach has worked for us in the past but now needs to change given our growing scale in employees and importance of your systems.</p><h4>Partners (Value)</h4><p><em>This evaluates:</em></p><ul><li><em>How many escalations have I had to intervene in between your and your partners — both personally and across your orgs?</em></li><li><em>Are you asking your most important partners for feedback, and are they willing to give candid feedback (of course knowing this could as much be on them not wanting to put effort in)?</em></li><li><em>Where there is tension with partners, have I received communication from you that there is misalignment, why alignment is difficult, and what you are doing to try and improve it?</em></li></ul><p>Example (Needs improvement at Director level): Everyone respects your judgment and willingness to work hard and do what is right for the company. On the flip side, as the examples that have escalated to me the last 6 months have shown, you can be quick to show frustration when you think someone is doing the wrong thing, often escalating your concerns it to high leadership very quickly without first trying to personally and resolve them. Overall this is an area where for you I think “your greatest strengths are your greatest weakness” is true; you rely a little too much on your drive to bulldoze things through with clear short term success, but its costing you relationships in the long term — i.e. people are starting to avoid collaborating with you.</p><p>Example (Exceeds at Director level): It is clear that you have built strong personal relationships with your peers that has brought your team some trust and time to build what is needed. You should continue to do so, with the main warning being that this way of keeping the peace is O(peers) usage of your personal time, and the number of directors at this company will steadily grow, as will their team size and they will become less in touch with their team’s unhappiness until it boils over. So you still need to invest in other more scaleable mechanisms.</p><h4>Company (Value)</h4><p><em>This evaluates:</em></p><ul><li><em>Have you spent time to add value to the business outside your direct scope</em></li><li><em>Are you leveraging that time to find its most valuable use, as opposed to falling back on things that we could achieve similar outcomes if done by someone more junior?</em></li></ul><p>Example (Needs Improvement at Sr Manager level): By nature you direct all your time and focus into activities directly related to your team. That was fine at line manager level, but now you need to find a way to create leverage across the organizaiton. We have talked about a couple of examples here such as taking over (and improving) my orgs weekly reliability meeting, which you have been a critic of. Next step for you is taking something like this, and showing you can lead it.</p><p>Example (Exceeds at Sr Manager level): Between the external talks you do and the fact that you lead the hiring guild for my entire org, you are clearly more than carrying your weight here in terms of leveraged impact.</p><p>Example (Exceeds at Director level): A natural strength for you, shown by your leadership of our coding interview squad, participation in giving multiple outside talks aimed at company wide hiring, and finally being a site lead. If anything this is one where I worry you are spending too much time vs challenges in your own team, and so going forward I recommend you find a way to delegate the interview squad to a staff engineer to lead it forward beyond you having so much involvement.</p><p>Example (Achieves at Sr Director level): This is a tough call as you are on the company wide incident commander rotation, which is a massive time sink for a busy leader of leaders, and until recently there was no real option for you to get off. So to clarify, I think given the limitations of the program you did the right thing staying on the rotation, this was high leverage value to the company, and if I was judging purely on merit I would say Exceeds, as there are clearly peers doing much less. Thus, the only reason I rate achieves within this writeup is to make a point with that program finally being fixed, you now need to get off the rotation and start to make leveraged cross-org impact in an entirely different manner — far more leading than participating. Now between potential projects XX and YY I think you are well set up on exactly the type of work to be successful here, you just need to follow through to delivery, then find the next ones.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=232d27774d66" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Dear Ex-Bigco New to Smallco]]></title>
            <link>https://inowland.medium.com/dear-ex-bigco-new-to-smallco-a8ef224d4384?source=rss-81db999c97a7------2</link>
            <guid isPermaLink="false">https://medium.com/p/a8ef224d4384</guid>
            <dc:creator><![CDATA[Ian Nowland]]></dc:creator>
            <pubDate>Tue, 30 Aug 2022 22:41:25 GMT</pubDate>
            <atom:updated>2022-08-30T22:46:57.351Z</atom:updated>
            <content:encoded><![CDATA[<p>We are excited to have you here! We put a lot of emphasis in our interviewing for what people bring to the table in taking us to the next level. Your experiences while at a BigCo are a large part of that. Again, we are really excited to have you here!</p><p>However. However, we have also found a decent number of Ex-BigCo people arrive here super excited to start making immediate organizational change, in a way that is non-productive and so when they try, turns out demotivating both for them and all the great people who have got us here. This document is my suggested path to avoiding that. Which is 3 things:</p><ul><li>Go read Camille Fournier’s<a href="https://skamille.medium.com/opp-other-peoples-problems-d7eb174724ee"> Other People’s Problems</a>, and internalize its lesson — take one problem to focus on, ensure you understand all stakeholders, fully deliver a solution, get the learnings and organizational credibility that comes from that, then onto the next. (A caveat to that is start small; credibility is a snowball you need to have iterated success on to drive iteratively bigger things).</li><li>In the meantime, try really hard in <em>shared</em> channels to avoid mentioning how you think your last company’s solution was much more sensible than ours and is just inevitable here. Even if you think you are just doing this as an interesting comparison.</li><li>Rather, focus on understanding our culture that has made us successful in our business, and gain the intuition for why it works, and why it should be challenged to change for our success ahead.</li></ul><p>Now, coming from someone with a senior title who has been here a good amount of time, this likely sound negative and demotivating after you just took a massive career risk. The flip side of it is Camille is not only a friend but was my manager for my last 18 months at a prior role at a fintech, which I joined after 10 years at a BigCo, and Camille wrote the OPP article 4 months after I left citing incompatible culture, so I am pretty sure I was on her mind. I especially know that as on my first day here, she had these coffee mugs sent to my new desk…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*B33PqUQS8BgwgWF-" /></figure><p>I.e. I speak from personal experience here.</p><p>Now the thing is, new ideas and new people’s passion are important, not only wanted, but needed to succeed at the next level of scale. Camille herself championed and has publicly praised my<a href="https://skamille.medium.com/my-experience-of-6-pagers-in-action-a16840cb8edf"> Amazon 6 pager lift</a> that I introduced to the fintech. But in hindsight, continuously making analogies <em>grated</em> on people who had been there longer<em>. </em>And I was not even making statements that Amazon’s way was better; I was just being comparative — even that grated. And I was just 1 ex-Amazonian in an org of 800; there are certain BigCo’s with much higher ratios both there and here, where you find they repeat the same analogies (“a monorepo will solve all our problems because they did at G/F”…). And with my experience here I can tell you each repeated “have you thought about X which worked so well at my last company” grates more than the last.</p><p>Why does it grate so much? I think there are 3 major reasons:</p><p><strong>It’s an Implicit Critique</strong> — Even just by virtue of bringing it up, there is a subtext that the status quo here is inadequate and the Bigco’s solution was likely better. That’s a great conversation in a 1:1 where it’s easy then to have an understanding conversation of tradeoffs, but not a really easy conversation to have in a group setting where you are challenging them in lacking the insight you bring.</p><p><strong>Not understanding the cost/benefit requirements of change</strong> — There are libraries of books written about the difficulties of organizational<a href="https://teambuilding.com/blog/change-management-books"> change management</a>. Every change to an organization sucks some amount of oxygen from the room for other changes to happen — at a certain point too much change is churn and the organization will reject it. Thus an organization needs to be thoughtful about making the highest leverage changes at any point in time. So when newcomers point out a problem pointing to a solution that clearly worked at their last company but seems like it will require a lot of work to implement, they usually lack context to being able to further answer “why this is the important thing our business needs us to come together to solve now”</p><p><strong>Not understanding the culture and the business ahead of a critique of its processes — </strong>Most organizational processes are path defined based on a company’s challenges and the culture that is needed to empower the right people to overcome those challenges. So if you come in criticizing processes, you are also criticizing culture. And this is a problem because the right culture for one set of business problems is not the right for all. I have become a big fan of Simon Wardley’s<a href="https://medium.com/org-hacking/pioneers-settlers-town-planners-wardley-9dcd3709cde7"> Pioneers/Settlers/Town-Planners</a> analogy for individuals’ preferred organizational behavior, particularly the idea that each “steals from” the last. What I have observed is that “stealing” often takes the form of forming alliances with others of the same type, that then “push out” the earlier type’s behavioral preferences, and so eventually ends up pushing out all people of the type. Now depending on the needs of the business (particularly the focus on execution needed to not fall apart as the number of stakeholders radically grows), that cultural change may be the right thing. But it’s as easy to be too early as too late here — go early and you lose the bottom-up discovery and creativity needed to build great products. So you need to make your argument for change bottom-up based on localized need at this particular time, as opposed to assuming a top-down, org wide solution.</p><p><strong>So in conclusion:</strong> We are happy you are here! We need to change to grow! Newcomers with less<a href="https://en.wikipedia.org/wiki/Normalization_of_deviance"> normalized deviance</a> and a lot of passion will be a big part of that! But it can easily go awry for the individual and the company, so watch and learn, seek to understand, pick your battles, and put personal skin in the game in resolving them!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a8ef224d4384" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Managing Systems Engineers]]></title>
            <link>https://inowland.medium.com/managing-systems-engineers-33e14e6c2ce5?source=rss-81db999c97a7------2</link>
            <guid isPermaLink="false">https://medium.com/p/33e14e6c2ce5</guid>
            <category><![CDATA[engineering-mangement]]></category>
            <category><![CDATA[systems-engineering]]></category>
            <dc:creator><![CDATA[Ian Nowland]]></dc:creator>
            <pubDate>Mon, 18 Jan 2021 23:38:35 GMT</pubDate>
            <atom:updated>2021-11-18T18:42:05.484Z</atom:updated>
            <content:encoded><![CDATA[<p><em>(or devops engineers, or production engineers, or systems development engineers, or SRE-SE, or cloud engineers, etc)</em></p><h3>1. Introduction</h3><p>The purpose of this post is to talk about how I have learned to appreciate the need for Systems Engineers, and what I have learned in terms of managing them well (both hiring and motivating). I do this because I think throughout our industry they are mismanaged, and a big part of this is that people from software engineering backgrounds tend to look at them as glorified toilers/scripters that can’t write production quality software but are useful for doing all the stuff software engineers complain about doing. I know this personally as this is exactly what I thought, before I worked with two great Systems Engineers on the early years of the <a href="https://aws.amazon.com/ec2/nitro/">Nitro project</a> and saw what game changers they were for our outcomes. They very literally did things in minutes that would take my software engineers hours, and could do it when it was much better for the business to have it done in minutes.</p><p>Now an important thing to get out of the way first — “why not just hire software engineers”? I actually agree for as long as it works for you, in <a href="https://inowland.medium.com/minimize-functional-hierarchies-in-agile-software-development-8b8889575999">my last post</a> I talked about minimizing functional specialist hierarchies, but I have a stronger opinion which is minimizing specialist engineering job functions for as long as possible. There are just so many advantages of role fungibility when you need to be highly agile to the engineering needs of the day. In fact, in my first thirteen years in industry before Nitro, I believed in hiring only generalist Software Engineers who could range all over the stack doing all the work needed — front end, back end, distributed systems, database administration, ops scripts. I still think that’s where you should start and hold the line on as long as possible for any individual team— i.e. push back when your software engineers start complaining “I don’t want to do this type of work” until it’s clear a specialist in that type of work is worth the costs.</p><p>However, as per the Nitro example, at a certain size of company and so complexity of problems being solved, specialized knowledge allows better solutions, delivered faster. So as engineering orgs grow big it does make sense to create specialties, although:</p><ul><li>doing so should be delayed as long as possible,</li><li>even then they should only be hired in places you really benefit from them</li><li>and as per the last post you should minimize hiring them into a hierarchy around function.</li></ul><h3>2. Work Process Definitions</h3><p>I will come back to other limitations of these definitions, but in terms of type of work, I will just restate the Google’s SRE Community <a href="https://landing.google.com/sre/sre-book/chapters/eliminating-toil/">great definitions</a>.</p><p><strong>Software Engineering</strong> — involves writing or modifying code, in addition to any associated design and documentation work. Examples include writing automation scripts, creating tools or frameworks, adding service features for scalability and reliability, or modifying infrastructure code to make it more robust.</p><p><strong>Systems Engineering</strong> — Involves configuring production systems, modifying configurations, or documenting systems in a way that produces lasting improvements from a one-time effort. Examples include monitoring setup and updates, load balancing configuration, server configuration, tuning of OS parameters, and load balancer setup.</p><p><strong>Toil</strong> — manual, repetitive, automatable, tactical hands-on system changes such as: manual releases, manual regression testing</p><h3>3. Doesn’t “the Cloud” Address Systems Engineering Needs?</h3><p>Nitro was of course in a space where there was unique systems complexity in building planet scale cloud infrastructure. In theory for its users, “the cloud” was supposed to make such complexity go away. I am now 4 years into using AWS rather than building it, and have learned what everyone in the same situation knows, which is given the complexity of the cloud offerings combined with the complexity of open source software that developers leverage, if anything there is more systems complexity then there used to be.</p><p>Now, there are many bad decisions that company’s and their software engineers make that needlessly but greatly add system complexity. The types of rules that avoid this are:</p><ul><li>Prefer Serveless to Containers</li><li>Prefer Containers Over VMs</li><li>Prefer vendor hosted offerings to running it yourself</li><li>Particularly for the last one when it involves running open source distributed systems</li><li>For the love of god, if you have 100+ engineers writing stateless web services, don’t make them configure Kubernetes themselves.</li></ul><p>The last one is an important one, as more generally you often can find ways that all the system engineering problems go to one org — an Infrastructure (or Platform) org, who builds nice shims that abstract away the systems complexity for a bunch of common problems. This often leads to all the Systems Engineers being in that org, and they have a slightly different hiring process, and the rest of the company moves on treating all systems problems as ones the Infrastructure needs to solve for the commons. Further it makes sense not to hire Systems Engineers outside that org for one-offs as it can encourage bad decisions around shadow infrastructure— particularly inexperienced developers/managers who chase the shiny new open-source/vendor toy of the day and then 12 months later their Software Engineers don’t want to operate it anymore.</p><p>The flip of this is, if you (like my employer, <a href="https://www.datadoghq.com/">Datadog</a>) are trying to solve some super hard large-scale systems problems while running on the cloud, then how your software handles that complexity is often not shimmable (in fact usually you see the shims make it worse). And that is when you struggle the most with a mindset of “Systems Engineers should only be in the Infra org” (or as an adjunct, our SRE/devops org). As per my last post, I think as long as your management learns how to manage them, you should hire them as close as possible to where the other engineers are working on other aspects of these systems and problems.</p><h3>4. A Brief History of Systems Engineers</h3><p>I want to summarize some history about how much this role has been re-named, as I do think its history of being undervalued is part of that. It actually starts with a different term, <a href="https://en.wikipedia.org/wiki/System_administrator"><strong>Systems Administration</strong></a>, which has a long history in computer systems, being the work of making software reliably solve real problems for real humans on real computers. As any old school systems administrator will tell you, this did not mean they did not write code. But that code did tend to take the form of shell and later Perl scripts, more focussed on harnessing the generalized software for a specific task, as opposed to being a larger piece of “software”.</p><p>Across the early-aughts, 3 things happened: software got more complex, software got delivered to computers more often, and the count of computers delivered to got much larger. From that the role tended to get renamed to <strong>Systems Engineering</strong>, reflecting that success in the face of increased complexity needed to be more proactive in building ahead of problems.</p><p>Across the late-aughts in the face of further growth, the role saw two variants gain success, <a href="https://en.wikipedia.org/wiki/DevOps">DevOps Engineering</a> (particularly coming out of fast moving companies using lots of open source), and <a href="https://en.wikipedia.org/wiki/Site_reliability_engineering">Site Reliability Engineering (SRE)</a> coming out of Google. In essence these were taking systems administration to more scale, the words of the founder of Google’s Site Reliability Team Ben Treynor were, “what happens when a software engineer is tasked with what used to be called operations.” Both DevOps and SRE have suffered through ambiguity in role and function.</p><p><strong>Devops</strong> — has a blurring of whether it was taking a software-centered approach to systems/operational engineering all done in a separate “devops” team, or a combination of both functions in teams of “you build it, you own it, you run it” style (the latter some now call <a href="https://medium.com/better-programming/devop-noops-difference-504dfc4e9faa">No-Ops</a>).</p><p><strong>SRE</strong> — has the challenge of whether it is just a bunch of good practices <a href="https://blog.stackpulse.com/site-reliability-engineering/site-reliability-engineering-sre-what-why-and-5-best-practices/">Google had refined</a>, or whether it is further a different discipline/function of software development where you needed a structural org hierarchy to <a href="https://sre.google/sre-book/being-on-call/">weaponize being oncall</a> to empower the right tradeoffs against feature/product oriented engineers and management chain.</p><p><strong>Both</strong> — have struggled with how much and complex software engineering is important for outcomes — is success 1000 lines scripts or 30,000 line programs? And how does that turn into hiring bars, promotion decisions and the like? In fact for a long time <a href="https://rachelbythebay.com/w/2011/11/22/caste/">Google was deliberately ambiguous that they answered this with 2 different hiring processes</a> both under the “SRE” label (i.e. SRE-SE vs SRE-SWE)</p><p>After these roles, in 2010+ you see some new names, with Facebook introducing <strong>Production Engineering</strong>, and Amazon introducing <strong>Systems Development Engineering</strong>. In my mind this is just rebranded Systems Engineering. But I know first hand in the case of Amazon at least, it was not at all a cynical rebranding, rather it was a deliberate decision to ensure Systems Engineers were appropriately recognized for how important their specialization was for solving complex systems problems at scale. Finally in the last few years you see some specialities within the speciality, like <strong>Cloud Engineers</strong> or <strong>Data Reliability Engineers</strong>.</p><h3>5. Understanding Systems Engineer Motivations</h3><p>As I have <a href="https://inowland.medium.com/management-area-4-people-subpart-2-managing-to-motivations-148244bbb0d9">said elsewhere</a>, the most important part of successfully managing people is understanding their motivations so you can manage to them. This is where most managers from a software development background struggled just like I did — because systems engineers do tend to code less, the assumption you make is you assume they will be motivated by working on simpler problems. But the opposite is the case. To understand that, I need to talk about categories of work again, but along a different dimension — how motivating they are.</p><p>While I like Google’s definitions around distinct grouping of types of work, I do not believe this work difference maps to the major differences between System Engineers and Software Engineers in terms of ability, skills, education, or training. Rather, I define these categories of work that modern agile “develop software and operate it” company’s need:</p><p><strong>Toil</strong> — as above — manual, repetitive, automatable, tactical hands-on system changes such as: manual releases, manual regression testing</p><p><strong>Support</strong> — on-call debugging production issues, dealing with user issues. Main difference to toil is it is inherently non-automatable; it requires either software or systems engineering judgment at least some of the time.</p><p><strong>Automation</strong> — Work to reduce toil. Since it reduces toil, included in this category is release engineering — i.e. investment in automated CI/CD pipelines (including coding the CI tests), and investment in building active monitoring canaries.</p><p><strong>Re-engineering</strong> — design/architecture/coding/delivery of changes to existing systems to run better in production to improve reliability at increasing scale.</p><p><strong>Features</strong> — design/architecture/coding/delivery of new features.</p><p>Importantly, all but Toil require both Systems Engineering and Software Engineering work. However, while it’s a spectrum and so I am talking about differences on average, I believe there are two broad groups of engineers who have very different motivations in which of this list they are motivated to work on. Starting with Software Engineers, I have found most highly index their motivation from the <strong>“</strong><a href="https://link.springer.com/article/10.1007/s10270-017-0621-x"><strong>flow</strong></a><strong>” </strong>experience coming from writing challenging code, and so they get unhappy unless they get to write some amount of challenging code that usually comes from building features and re-engineering (although large scale automation has it as well).</p><p>Whereas those who specialize as Systems Engineers are much more motivated by three other things: bringing things into order, understanding broadly and deeply how complex systems work, and seeing their work have an impact. This means Systems Engineers are more motivated by re-engineering and automation projects, and even some of the hard support cases. But just like software engineers, they don’t like the following.</p><p><strong>Toil — </strong>Engineers are builders. Toil is not building. No engineer likes large amounts of toil. Systems Engineers will do their share, but toil is no more interesting or motivating to them then it is to Software Engineers.</p><p><strong>Simple System Engineering — </strong>Systems Engineers need growth in their work. If 5 years ago a Systems Engineer could have written this simple YAML you want churned out across 40 similar but slightly different microservices, expect them to do it out of duty or expediency, but that’s different from being motivated by it. I bring this specific example up as Automation is the one specific area software development managers first hiring Systems Engineers expect it to be 100% of the job. It can be if it has complexity, but that is usually not the case if you need it to be 100% of the role.</p><p><strong>Growing technical debt — </strong>No matter the challenge of the work, Systems Engineers are demotivated if their work makes long term systems complexity worse. Now this gets into a discussion of technical debt, with engineers viscerally arguing it is inherently bad as it immediately makes their job harder, whereas most managers know that if it is taken on judiciously it can be right for the business, especially in a fast headcount-growth environment. While the managers are right, technical debt is hard to make its future costs visible, and so it runs into the problem that management finds it hard to get the balance right, particularly when paying the balance is based on future headcount which may not arrive. Now this is all true of Software Engineers as well, but the reason this is more acute for Systems Engineers is:</p><ul><li>Managers from Software Engineering background often lose their visceral understanding of the tradeoffs when it applies to Systems Engineering work.</li><li>Systems Engineers are often asked to do things like region buildouts, new vendor buildouts etc which are often asked for on extremely tight timelines, and so corners must be cut.</li><li>Systems Engineers have a different view of systems complexity to a software engineer — “lets introduce a new distributed open source storage system for a 25% performance win” can sound really motivating to the Software Engineer building it, whereas the System Engineer know what the maintenance story is going to look like in 12 months.</li></ul><p>In any case, just like Software Engineers, Systems Engineers will spend some amount of time doing projects in all three of these categories, just expect them to be ground out if it becomes the majority of the role for a long period of time. (Also expect it to be hard to close them unless you can show you understand this in the interview process).</p><h3>6. Hiring Systems Engineers</h3><p>Stating a well known but little directly discussed point: a lot of Systems Engineers struggle to pass the coding questions done in the interactive style (i.e. whiteboard, but even full IDE if under a time limit with people watching them), even though they can write code on the job. So the thing I want to cover here is why I think that is, and how to evaluate their coding ability in light of it. One thing to get out of the way first, is it’s clear you need to evaluate coding ability in the hiring process. There are a lot of people who are extremely competent at doing complex work in their profession, but cannot learn to write anything beyond trivial code. I myself have a Mechanical Engineering degree, and was perplexed watching otherwise very strong peers who couldn’t write 10 lines of MATLAB from scratch. But it is very clearly true. So since there’s a need for it and some people won’t be able to do it, you need to evaluate it.</p><p>But how to do so? I think the first thing in answering is taking a step back and looking at why exactly Systems Engineers are less motivated by the “flow” experience of coding that Software Engineers love so much. In my discussions I have heard 3 answers, which I think are somewhat related:</p><p><strong>Dopamine-seeking — </strong>I won’t say ADHD as it’s a clinical term, but some struggle to focus on one thing for 3–4 hours that a lot of deep software development work requires, particularly when on large codebases so you have to build then hold a lot of detail in your head.</p><p><strong>Perfectionism — </strong>The best Systems Engineers I know have an encyclopedic knowledge of detail that they bring to a problem, but it means when they come to transforming it into code, they recoil at the tradeoffs they have to make and so get stuck doing <a href="https://en.wiktionary.org/wiki/yak_shaving">yak shaving</a> activities rather than shipping and iterating.</p><p><strong>Dislike of abstraction — </strong>the usual complaint Systems Engineers make about Software Engineers is their over-use of abstraction. This is fair. On the flip side, the only way you build large software is with abstractions, which even <a href="https://www.joelonsoftware.com/2002/11/11/the-law-of-leaky-abstractions/">as they leak</a> (and I think it’s those leaks that Systems Engineers hate the most) allow layering to build bigger things.</p><p>So what’s the point of this in answering how to interview them? The point is whiteboard coding interviews take all 3 and artificially ramp them up in a manner completely unrelated to the job:</p><ul><li>Interviewer looking over your shoulder distracting you from thinking deep</li><li>Toy problems where you’re supposed to intuit from the context what the complex enough solution shows that they are smart and code, but not too complex that they get stuck in a rathole and don’t write much code.</li><li>Heavily focussed on pure-CS abstractions unrelated to day to day work</li></ul><p>Which this in mind, I think best practice is:</p><ul><li>Small practical take home problem which should take 1–2 hours to solve</li><li>Since there is some risk of cheating, ideally validate using open source contributions.</li><li>If they don’t have that, at the in-person stage, validate with a ~15 minute non-abstract, simple coding question (&lt; 10 lines).</li></ul><p>Finally, I don’t want to say Systems Engineers are worse in this area than Software Engineers as I have seen plenty of bad both ways, but I will say in terms of knowledge type questions, be very weary of mid-career interviewers assuming that particular things they know as must-haves for someone coming in to the role, particularly at senior levels. I have heard plenty of <a href="https://en.wikipedia.org/wiki/Narcissism_of_small_differences">narcissism of small difference</a> stuff on why given a candidates current knowledge they were not a fit for a specific Linux userspace vs Linux kernel vs Postgres vs Networking vs Reliability practices vs Containers vs Cloud vs Distributed Storage vs In-house distributed systems vs etc role. If someone is strong in other areas (i.e. has shown they can go deep) where they should be strong, and has good general aptitude otherwise, and will have a good mentor, none of this past knowledge stuff matters much after about 6 months. In fact they may be more effective as they are more motivated by learning.</p><h3>7. Why Not Call them Software Engineers?</h3><p>This is an important one for me to cover as it comes up with some of the engineers at my current company. I understand where this is coming from, especially given the history of the role being treated as a second class citizen, made worse because any individual is a blend and not as black/white one or other as I used in the rest of this document. As stated in the intro, I think you should avoid specifying functional specialities as long as possible. However, that fungibility comes with a lot of difficulties in hiring and promoting people who aren’t perfectly fungible and have unique specialties. Now in the short term, you can get away with generalizing your job levels as much as possible, and then letting one high need org have a slightly different bar and using their local trust to keep the bar. However, outside that org the informal bar can cause a lot of bad blood around standards, things like:</p><ul><li>“That other org is easier to get promoted than ours”</li><li>“That other org has shitty coders”</li><li>“That other org is a bunch of middleware coders who aren’t deep enough in systems knowledge for the problems they are solving and are going to create a mess”</li></ul><p>A classic example of this is hiring a 15 year systems engineering expert as a “mid” level role as they didn’t crush the coding interview like someone 5 years out of their CS degree who you made “senior” did, even though the latter treats everything outside the JVM as someone else’s problem. (or not promoting said veteran because “the complexity of code in their delivered solutions does not hit our senior engineering bar”).</p><p>”Calibration” is one of those words that sounds like shitty corporate-speak but if you have ever been in an organization that struggles with shared language and understanding of hiring and promotion bars, you realize how essential a formalness around expectations is needed otherwise you end up with pretty bad infighting over it. In fact the <em>only</em> thing I have found that is more “political” for both engineering and management is questions of fairness around headcount and recruiting access. So there is no avoiding that as company’s get large, calibration of role expectations across larger and larger groups becomes important, and it greatly helps those to be able to talk about different expectations of people working in different functional specializations. Which means a different name.</p><p>None of that is to say you need to use “Systems Engineer” as a name, in fact it has a downside, in that we have already seen Software Engineers should do Systems Engineering work, and vice versa. As I have said, Amazon uses “Systems Development Engineer” to acknowledge the role’s creative aspect. Facebook of course uses “Production Engineering”. My prior company used “Platform Software Engineer”. I have also heard “Infrastructure Engineer” although that has the downside of usually also being an org name. The main one I suggest to stay well away from, is variations of “Reliability Engineer”. This is because it creates a needless division firstly that the job is always about reliability, and secondly that reliability isn’t an important part of Software Engineering’s mission as well. This is deeply ironic because Google, the creators of SRE themselves split up the function within their org. This is something I will go deeper into in my next post.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=33e14e6c2ce5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Minimize Functional Hierarchies in Agile Software Development]]></title>
            <link>https://inowland.medium.com/minimize-functional-hierarchies-in-agile-software-development-8b8889575999?source=rss-81db999c97a7------2</link>
            <guid isPermaLink="false">https://medium.com/p/8b8889575999</guid>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[engineering-mangement]]></category>
            <dc:creator><![CDATA[Ian Nowland]]></dc:creator>
            <pubDate>Tue, 12 Jan 2021 01:21:37 GMT</pubDate>
            <atom:updated>2021-01-16T01:41:52.527Z</atom:updated>
            <content:encoded><![CDATA[<p>TLDR: Each functional hierarchy costs you agility at the square of total count, and there is a lot of functions in agile devops software development.</p><h3>1. Overview</h3><p>Over the last 20 years, the software development industry has faced two independent, similar but sometimes conflicting revolutions fundamentally about bringing together different functional expertises (“functions”) together to collaboratively rapidly iterate via tighter feedback loops -</p><ul><li>the <a href="https://en.wikipedia.org/wiki/Agile_software_development">Agile</a>¹ movement embracing cross-functional teams taking an iterative approach to delivery</li><li>the <a href="https://en.wikipedia.org/wiki/DevOps">Devops</a> movement pulling together software development and operations functions into to improve addressing operational issues by fixing core software.</li></ul><p>Out of this union comes two questions around organizational structure:</p><ul><li>How to structure the organization around functions who are important to the development and operation of software, but lack the technical expertise to be part of a devops team?</li><li>For those functions within the development and operation of software, how to structure the different software and systems engineering specialties?</li></ul><p>Today, there is no de-facto standard across companies or even within companies across functions. Steven Sinofsky has the <a href="https://medium.learningbyshipping.com/functional-versus-unit-organizations-6b82bfbaa57">state of the art writeup of tradeoffs for software development</a>, but as it is based on his experience in Windows and Office which were neither devops nor agile, not all of it carries over for those doing both. Thus, in the rest of this document I attempt to lay a specific proposal for companies (going forward “agile software development” companies”) who seek to get the best of both worlds.</p><h3>2. Organizational Structures Background</h3><p>There is a large amount of literature on tradeoffs of different organizational structures of companies. One of the important distinctions is whether the structure is <a href="https://online.pointpark.edu/business/types-of-organizational-structures/">Divisional, Functional or Matrixed</a> (with divisional sometimes called “Unit”). Note some literature talks about these as being absolutes at the company level, yet most companies are blended in implementation, depending on the specific function.</p><p>Most agile software development companies follow a functional structure for functions significantly removed from the creation and operation of software. That is functions like Legal, Finance, HR, and Sales are all hierarchies that report up to the CEO. Now a caveat for massive companies is some of these are broken to the mega-divisional level (say AWS having its own Finance org) — essentially though this is just treating the mega-division as a sub-company.</p><p>Caveat aside, because these functions are predictable and well regulated in what their outputs are relative to what is needed by other functions, them having a functional hierarchical structure is widely viewed as the best option. So as an example, while many people may wish their company’s legal team was more rapid to respond to requests, that is different to company’s needing their legal organization to be more agile in iterating with other functions to solve large complex problems.</p><p>All of that is in contrast to the roles involved in developing and operating software. The functional specialties here are diverse: Product Management, Program Management, Design, Usability, Support, Operations, Data Science, Security Engineering, Software Engineering. The last itself has many sub-functions which large company’s treat as specialist functions, for example: Front End Engineering, Distributed Systems Engineering, Systems Engineering, Reliability Engineering, Data Engineering, Systems Software Engineering. Across both lists, complex coordination between individuals across functions is very important for success of a project/product, and so highly functional parallel management orgs do not work. On the flip side a lot of cross-system, cross-teams and often cross-product communication <em>within</em> function is necessary as well, so pure divisional structures do not work either.</p><p>Appearing in the 70’s and 80’s, one solution proposed to this for technology companies is matrixed management, where functional specialties have a management tree (under “functional management”), and then that matrixed into a project tree (under “project/program management”). This was first used at NASA, Andy Grove implemented it at Intel (as documented in <a href="https://www.amazon.com/dp/B015VACHOK/">High Output Management</a>), and it is still in place today at companies such as Qualcomm. Those are all notable successes, and I am sure it is still used today at many more companies where physical manufacturing is an important part of the innovative process. Still it has had its failures, for example <a href="https://odinnovator.com/restructuring/demise-nokia-cautionary-tale-restructuring-gone-wrong">Nokia</a>. That article is well worth reading as it highlights both the particular failure but also makes a meta-statement about matrix-management — the structure itself is heavyweight to implement at scale. And so while for large, unagile, slowly iterative technology industries with manufacturing overheads the structure can make sense, for an industry that has embraced iterative release like agile software development, all that structure seems a burden. Indeed the only company I know who have documented trying, Spotify, <a href="https://www.jeremiahlee.com/posts/failed-squad-goals/">admits it was a failure</a> mostly due to the overheads of the management structure.</p><h3>3. Common Structure in Agile Software Development</h3><p>The last section raises, what do agile software development organizations do? Here most companies start with a <em>core</em> software-development function splitting itself into a divisional hierarchy along the lines of the technical architecture²:</p><ul><li>Division (say under a VP)</li><li>Product Group (say under a Director)</li><li>Product (say under a Manager)</li><li>Component/service (say under a Team Lead)</li></ul><p>Then for the other functions, you see three options:</p><p><strong>Cross-Functional </strong>— individuals in a specific function are cross-functionally placed into core software development teams. For example, a Front End Engineer or Reliability Engineer on a team of Backend Software Engineers working on an Alerting product.</p><p><strong>Mini-Matrix </strong>— They have functional teams, but any single team is pushed down to the lowest level of the core hierarchy that supports their mandate in terms of cross-functional collaboration, which allows the “matrix” aspect to be kept informal. For example, a team of 5 front end engineers under a Director, 2 work on an Alerting product, 2 work on a SLO product, and 2 works on an Incident App product, each of those 3 products having their own backend development teams of 5–6 people. Or even a company-wide Architecture team, time slicing their functional expertise across all teams in the company.</p><p><strong>Functional Hierarchy</strong> — Functional teams are formed into a nested and parallel hierarchy reporting somewhere up in the core hierarchy. For example, a Front End org, or a Reliability org, reporting to a VP, SVP, or sometimes even the CEO.</p><p>As stated in the intro, beyond this what any company chooses tends to be unique to both the functions and the company’s mission (and organizational debt³ in light of mission).</p><h3>4. Choosing Model For Specialty Functions</h3><p>My basic rule comes from a combination of three things:</p><ul><li>The goal is the agile delivery of software that is also operated, which is a problem of cross-functional and cross-divisional complex coordination</li><li>In section 2 I laid out at least fifteen different specialist functions involved in the delivery and operations of software</li><li>Closeness in the hierarchy makes complex coordination easier</li></ul><p>The third point has a strong underlying assertion that is important to address; namely, that management hierarchies create<em> </em>a power structure with the downside of making collaboration harder the further apart individuals are in it. This assumption underlies all management literature about why organization matters and so seems a truism. However, advocates for functional hierarchies often argue that they are not looking to create a power structure, but rather want to solve other important problems. Those problems include things like:</p><ul><li>Ensuring consistent hiring into the functional role</li><li>Ensuring that promotions are evaluated by peers who understand excellence in the functional role</li><li>Making it easy for the staff in the functional roles to float between teams as their skills are needed</li></ul><p>However, all of these requests can be met with other mechanisms. Interviews can be conducted and reviewed by functional experts, functional committees can review promotions, and it is possible for teams to occasionally lend out experts to other teams where there is a need for a critical project<em>. </em>Creating a functional team makes these goals easier to achieve <em>because</em> it creates a formal hierarchy and grants power to the leader of the team. The functional team to solve these problems is particularly seductive early on in its existence, because when it is created everyone tends to agree on their priorities on what they should work on and why. But over time, this agreement will fade. The company will grow, and people will be removed from the founding relationships. Newcomers to the team will assume that this team operates in the same way other teams operate: where the most important relationships are inside of the team itself, followed by other nearby teams in the reporting chain. As this happens, the downsides start to occur more and more frequently, taking multiple levels of management to resolve, and turning into organizational debt.</p><p>With that in mind I propose the basic rule of “be extremely careful any time you create a new nested management hierarchy for a functional specialization”, which means the preference should be Cross-Functional, then Mini Matrix, and only finally Functional Hierarchy. Now, “Extreme care” does not mean never having a functional hierarchy. For example at Datadog we have a strong preference for a seamless user experience across product facets and even products. For us, that argues nesting the Design and Product Management hierarchies is worth the cost. But we do it for two specialities, not fifteen.</p><h3>5. Agile Software Development Management is Cross-Functional</h3><p>The previous section leads us to see that companies who want to succeed in minimizing parallel hierarchies need to find mechanisms of empowering specialized functions so their concerns are addressed. That only comes from changing the way the organization thinks about “software development management”. There is an old saying in organizations where non-core functions are asked to succeed through influence — “Lots of responsibility, no authority”. The flip of this, for those who have experienced middle management in rapidly growing agile software development orgs, is our roles are “lots of authority, no autonomy”. That is, we have a role which nominally has a lot of power, but out time and focus ends up being spent as the negotiator that makes everyone net maximally happy in light of divergent wants. Across upwards/downwards/sideways stakeholders, both in-function and cross-function. In fact, this can feel like constantly owning “no-one is happy” tradeoffs, but that is a feature, as the other option is having functional mini-empires whose leaders feel great about their tradeoffs, apart from their fights with other mini-empires who have different opinions, and nothing is resolved.</p><p>Thus, in the type of structure advocated for here, the company has to understand “software development management” above “line” team lead level is not about software development, nor is it only about “people management”. This is why those are just two of the seven areas of <a href="https://medium.com/@inowland/the-seven-areas-of-software-management-be89d213ea7">Software Development Management</a> I think take equal weight. In particular there are two areas that managers need to spend significant time on for this type of structure to work — Partner Engagement and Company-Wide management processes. This means spending significant time/focus on problems such as “the 10% of my team who are front end engineers want to feel like they have career development prospects”. Needless to say without both coaching and rewarding such behavior at the highest level of leadership, it won’t magically happen.</p><h3>6. Addressing Concerns of Non-core Functions</h3><p>As I stated in section 4, a lot of the initial pushers for functional hierarchies do so with legitimate concerns of problems within the company that a functional hierarchy would address. The point is that the future costs of a functional hierarchy are very high, and there are other options. So in this section I lay out common concerns and how to address them.</p><h4>6.1. What about standards of recruiting and promoting?</h4><p>This is a heavily recurring concern, and is often the thing that growing software development organizations do too late for their specialist functions that lead to those in the function feeling like second class citizens and so wanting to centralize. My rule of thumb is when you have ~10 people in a function company-wide, you need to invest in level matrices and standardized processes for both recruiting and promoting, and they need the time and focus of Recruiting, HR, Engineering Management, and senior ICs to be made legitimate.</p><h4>6.2. Function X is extremely important in our culture/business/etc?</h4><p>As covered in section 4, there are legitimate reasons for breaking out some functions in their own hierarchies to suit the culture and problems of a particular business. However with each, you are making it easier for co-operation internally in that function at the cost of impeding cooperation with other functions, and this impedance rises as a N² problem. This can be particularly impactful as this is a slippery slope: when a function centralizes, there is always a next in line who can use most of the same arguments about why they need to centralize too. So in general, the best solution here is centralize into teams at the lowest place in the chart that makes sense. But don’t centralize company-wide unless you really, need to empower that function to address a core challenge of your business.</p><h4>6.3. Function X is something our core software development org does badly?</h4><p>This is an argument often used within the engineering functions; e.g. “we are bad at making our systems reliable”, “we are bad at building great front ends”, “we are bad at building scalable back ends”. These are often truths that come about based on the majority subfunction of engineers the company hired in the past and has now promoted into management. The key point is, putting a speciality function into its own hierarchy makes it easier for them to work with each other <em>at the cost</em> of making it harder to do so with everyone else. Now, if you need that, and can live with the consequences for the long term, then centralizing makes sense (say a Mobile development org). On the other hand a lot of times you need these agile coordination problems solved throughout the organization, and at that point you are better off finding better ways to empower the speciality without it needing to be a hierarchy.</p><h4>6.4. What do I tell smaller functions about their limited growth options?</h4><p>The first point is, a second level of hierarchy of say manager of ~5 leads of ~5 ICs, is a 1 in 30 role. While growth opportunities for the 1 in 30 should not be neglected, you need to be thinking about growth opportunities for the 29 others much more in their roles as individual contributors or of closely managing individual contributors. That comes from 3 things:</p><ul><li>Greater functional technical expertise through increased depth and breadth of hands-on experience</li><li>Greater social expertise allowing them to execute their functional expertise on projects/programs of greater risk, scope, ambiguity, complexity and so impact.</li><li>Greater ability to impact the company through helping others to between execute and grow in the function.</li></ul><p>Now sometimes for a specific function an org can’t supply the opportunities for the second or even the first. There’s just not the business need. Centralizing into a hierarchy isn’t going to fix that, and worse may create a situation where opportunities get sought out contrary. However, if the business does have needs, then the only thing a functional hierarchy makes easier is to move people between projects/programs to maximize average growth. That is important but comes with costs, and the need for moving people between teams to maximize growth applies just as much to core engineering as other functions. Thus I argue you can meet this by holding management accountable for the growth of all individuals in all functions under them.</p><p>Coming to the third point, I would say this is a strength of not centralizing where it is too easy for work like this to be a “management problem”. Instead with sufficient guidance, this can be seen as part of the role of senior IC’s and have the second order effect of allowing them to build the social relationships and capital that also helps them succeed at 2.</p><p>Finally while it comes with costs, don’t assume you can’t mix other functional backgrounds into the core software development management tree; i.e. get past the “only ex-engineers can manage engineers” mindset. The strongest example of that is Amazon where leaders of any function can manage engineering teams provided they can demonstrate strong leadership in the face of teams core challenges, which often are not technical innovation.</p><h4>6.5. What about authority/advocacy of best practices of the function?</h4><p>As stated in the last section, this can be done without a functional hierarchy. However it does require management work and enablement of the community, examples I have seen:</p><ul><li>When you have 1–5 people who are clearly most senior in the expertise, ask them to do this as 50% of their role, often with a EA or TPM helping them on the coordination side.</li><li>When you have either too many or too few for that, I do think the more generalized notion of a <a href="https://businessandfinance.com/squads-tribes-guilds-agile-management-future-workplace/">“guild” that came out of Spotify</a> can work, but likely only with someone senior up the tree (say Director level or above) to spend the time and focus to enable it.</li><li>Should those not be possible, you can get away with a single “core team” (for example, a company-wide architecture team), although they are often limited by the fact that they are not community-based nor hierarchy-based in their source of power, so can struggle to have influence. To counter that I would suggest giving them a mission that still needs them to deliver as individual contributors, as per the next point.</li></ul><h4>6.6. What about common software/problems that need a team of specialists?</h4><p>At the team level, this is completely fine. If the common problems are big in terms of internal coordination, there is no trouble with nesting either — this is just a “platform org”, which is common in core engineering. Further, platform teams are often a great place to put the most experienced IC’s, drawing on their expertise technically and organizationally to allow advocacy of building the right things, and so also acting as an anchor of authority and advocacy.</p><p>The only warning here is a more general warning about platform organizations, which are infamous for saying “we will solve the problem you have today in a more general way in about 12 months” — i.e. they can end up heavily favoring throughput over agility as they are removed from the advantages of agility for the broader business as they only see its costs. My last manager Camille Fournier has a good writeup of the <a href="https://skamille.medium.com/product-for-internal-platforms-9205c3a08142">techniques</a> you need to use to counter this.</p><h4>6.7. What about flexibility for moving people to where the business needs it most?</h4><p>It is clear having all specialists in the function report up to a single point makes it easier for that point person to move people between their teams. If this is a major concern you can see the business following through on, this more or less falls into what section 6.2 describes about this being an important exception. However, be sure the follow through is likely. All of these functions are high skill, projects usually take months, and specialist knowledge of a particular domain can take years to learn. Are you really going to move such functions around rapidly and on mass? In my experience I have only seen this exercised regularly for lower-skilled functions, much less involved in the agile delivery and operations of software — really it’s only been for un-agile pre-devops operations teams.</p><p>[1] I use “Agile” with trepidation, as what started with <a href="https://www.amazon.com/gp/product/0131467409/">good intentions</a> around rapidly-iterative minimal processes turned into an industry of consultants selling a whole lot of process. When I use “agile” I mean it the original definition, which came from software development getting over its “<a href="https://wiki.c2.com/?EngineeringEnvy">engineering envy</a>” at the processes and rigor used by the more physical forms of engineering, and instead embracing its virtual-ness in implemented form to allow late-binding in meeting and so specifying requirements, and so allowing quick iteration to solve complex problems at which all the other forms of engineering <a href="https://cliffberg.medium.com/spacexs-use-of-agile-methods-c63042178a33">now envy</a>.</p><p>[2] When done thoughtfully, this is called embracing <a href="https://en.wikipedia.org/wiki/Conway%27s_law">Conway’s law</a> around shipping the org chart by structuring around what you want shipped</p><p>[3] For lack of a better term, “organizational debt” is the heavy inertia of changing the structure of human organizations, which beyond the conflict of having winners and losers, is exacerbated by any proposed change having very clear losers and less clear winners.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8b8889575999" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Fortnightly Updates FAQ]]></title>
            <link>https://inowland.medium.com/fortnightly-updates-faq-ffc34c0035cd?source=rss-81db999c97a7------2</link>
            <guid isPermaLink="false">https://medium.com/p/ffc34c0035cd</guid>
            <dc:creator><![CDATA[Ian Nowland]]></dc:creator>
            <pubDate>Tue, 23 Jun 2020 23:41:07 GMT</pubDate>
            <atom:updated>2020-06-23T23:41:07.691Z</atom:updated>
            <content:encoded><![CDATA[<h4>What is a fortnight?</h4><p><em>(noun) BRITISH a period of two weeks</em></p><p>Used as “biweekly” <a href="https://www.merriam-webster.com/words-at-play/on-biweekly-and-bimonthly">is ambiguous</a>.</p><h4>What is the TLDR?</h4><p>Every 2 weeks we walk up the org tree, writing a short number of bullet points of wins and challenges for the last 2 weeks. By “Walk Up” it is meant at each level up, the most important, interesting and/or impactful points get reused, although often rewritten for a broader audience.</p><h4>What is the purpose?</h4><p>Firstly, trying to find a lightweight, regular way for everyone to have an appropriate amount of information about what other teams are doing. Secondly for some people/teams, it should work as a nice record of “what really went on that quarter”. Thirdly it is a mechanism for forcing managers to regularly account for “what really happened the last 2 weeks”, in a way that sprints (and otherwise) fail to do.</p><h4>Do I have to stick to a hard format?</h4><p>At the highest level, yes for consistency across a broad audience. At a lower level, doing your own thing is fine, e.g. doing it weekly, having more categories/text, etc. The main thing is having the information to share at the next level at the needed time.</p><h4>What is the process in detail?</h4><p>Every 2nd Wednesday, each line manager either:</p><ul><li>Knows from daily stand ups what each team member has been up to</li><li>Asks their team to send them an update for the last 2 weeks by lunch on Thursday, by either a shared document, email to the team, or email.</li></ul><p>By end of the next day, Thursday , each Manager takes the 2–4 most interesting/impactful wins and 2–4 challenges, augmenting with any they have themselves, and publishing them with a document that everyone under their Director can see.</p><p>Then by the end of the Friday, the Director takes the ~4 biggest Wins and ~4 biggest Challenges, augmenting with their own, and publishing them in a doc which everyone under the VP can see.</p><p>Then on the next Monday, the VP writes their own takes the ~4 biggest Wins and ~4 biggest Challenges, and sends that out, with a link to the full doc to:</p><ul><li>Their full org</li><li>Their peers, and the senior ICs/managers below them</li></ul><h4>What is a win or challenge?</h4><p>These should be 2–3 sentences, written so someone outside the team can understand what it is, and what is its impact. At the VP level-doc, most should be:</p><ul><li><strong>People</strong>: Someone leaving/joining team, being promoted. Team splits.</li><li><strong>OKRs</strong>: A KR hitting a key milestone, or being delayed.</li><li><strong>Operations</strong>: Acknowledging an incident or a bad week, celebrating a good week in terms of incidents or pages.</li><li><strong>Engineering/Product</strong>: Letting people know when a RFC or Product brief has been published.</li></ul><p>At levels below VP they may be more tailored towards specific milestones for a team, or individual. Examples of good writing are:</p><ul><li>Team X has now delivered feature Y. This was a big ask of customer Z, and we think this allows us to have much more success in market alpha going forward.</li><li>Team X has now fully migrated to the company wide platform Y. This involved many difficult challenges we have documented at alpha and beta.</li><li>Team X has deep diced on issue Y, and found that it delivers a Z% cost improvement across our full customer base.</li><li>Team X is having challenges hitting the company-wide deadlines in delivering Y due so specific issues alpha and beta. We have escalated this to the respective management, but are now doubtful to hit goal Z.</li><li>Team X has had multiple sev 1 operational issues the last week, all root cause issue Y. We are actively implementing z to stop the bleeding, and full long term post mortem action items is at alpha.</li></ul><h4>Can I have a template?</h4><p>This is a<a href="https://docs.google.com/document/d/1R25MRKsWq5_MihsLd9PgXV49_rF_1S4a8Y8ENt12O3A/edit#"> sample doc</a> for an org.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ffc34c0035cd" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Management Area 4: People, Subpart 2 — Managing to Motivations]]></title>
            <link>https://inowland.medium.com/management-area-4-people-subpart-2-managing-to-motivations-148244bbb0d9?source=rss-81db999c97a7------2</link>
            <guid isPermaLink="false">https://medium.com/p/148244bbb0d9</guid>
            <category><![CDATA[motivation]]></category>
            <category><![CDATA[engineering-mangement]]></category>
            <dc:creator><![CDATA[Ian Nowland]]></dc:creator>
            <pubDate>Tue, 20 Aug 2019 01:11:24 GMT</pubDate>
            <atom:updated>2019-08-24T01:08:56.590Z</atom:updated>
            <content:encoded><![CDATA[<h3>Management Area 4: People — Managing to Motivations</h3><h4>Why optimizing for business value is not the right thing for the business</h4><p>(Part Four, subpart two, of <a href="https://medium.com/@inowland/the-seven-areas-of-software-management-be89d213ea7?source=your_stories_page---------------------------">The Seven Areas Of Software Management</a>)</p><h4>Introduction</h4><p>The last post introduced why <a href="https://medium.com/@inowland/management-area-4-people-part-1-understanding-motivations-35beea0a8a52">understanding your team’s motivations</a> was fundamental to people management. That turns to the next problem, which is what motivates your team members is sometimes/usually different to what business needs would immediately have them do. Handling this is what separates great people managers from mediocre ones who churn team members every 1–2 years — realizing that when you disregard motivation, and just optimize for the business in the short term, then the business will not have the optimal outcome in the long term, as unmotivated people will move on, taking their hard acquired knowledge and experience with them.</p><p>The easiest way to talk about this topic is to focus on one area of motivation, and I will focus on Flow. The reason is as someone who has 8 years managing teams of ~75% software engineers, it seems to be most software engineers primary motivation. The reminder though is that not everyone’s motivations is the same, and managers need to be able to appreciate and manage team members with other motivations.</p><h4>What is Flow?</h4><p>The idea was studied and popularized by <a href="https://en.wikipedia.org/wiki/Mihaly_Csikszentmihalyi">Mihaly Csikszentmihaly</a>, in particular in his book, <a href="https://www.amazon.com/Flow-Psychology-Experience-Perennial-Classics/dp/0061339202">Flow</a>. Straight from Wikipedia: “flow state is an optimal state of intrinsic motivation, where the person is fully immersed in what they are doing. This is a feeling everyone has at times, characterized by a feeling of great absorption, engagement, fulfillment, and skill — and during which temporal concerns (time, food, ego-self, etc.) are typically ignored.”</p><p>Csikszentmihal has 9 conditions needed to achieve “Flow” state, which a lot of computer-centered work innately brings as long as the <a href="https://www.purplecrane.com/blog/posts/2018/07/13/flow-in-software-development">environment is well managed</a>. However, the hardest of the nine is that the work must have the right amount of difficulty balanced against each individual’s skill. “To achieve a flow state, a balance must be struck between the challenge of the task and the skill of the performer. If the task is too easy or too difficult, flow cannot occur. Both skill level and challenge level must be matched and high; if skill and challenge are low and matched, then apathy results.”</p><p>Diagrammatically:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/439/0*YTSCc8UM-9uhHzDv" /></figure><p>The challenge is that software development is high acquired-skill work — in solving problems and writing code engineers are acquiring more skills that makes the same problem less hard in the future.Thus, seen from the viewpoint of productivity, our best optimization is to have engineers work where they are most highly skilled. However, from the view of getting flow, once an engineer is highly skilled in a certain domain, it is less likely there are enough challenges left for them to hit a state of flow. And so long term thinking managers have to balance.</p><h4>Software Developers and Flow</h4><p>Stating what you need to do here is easy: to the extent you are reliant on deep acquired skills to get the right business things done effectively, then sometimes you need to let your team do things which are highly motivating to them, whether or not that is strictly the top immediate priority for the business. The way I visualize this is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/553/0*MFiRyW_DyOMQEgr1" /></figure><p>That is: good managers walk a curved line between balancing what the business says it immediately wants, and what each engineer wants to do. Importantly: on the picture, that the final point reached isn’t the original business ask is deliberate. The hope is by giving engineers some freedom you will end up with something better for the business than what it asked for. But it might also be worse. But it will never be the same, particularly in software systems where you often have the following dynamic of work needing skill (i.e. presenting challenge)challenge falling over its age:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/634/0*Ao36LHO1ZSUpcwj-" /></figure><p>i.e. Experienced developers will want to work on new systems much more than the business strictly needs. Now, just following that strategy can go awry — anyone who has worked in software development for a while can point to whole teams where they seem to deliver the bare minimal business value while concentrating on what motivates them — rewriting a system that does not need to be rewritten. That is a management problem, but it is foolish to think none of us will ever be part of doing it. The next section has the step by step to get the balance right.</p><h4>Step by Step Managing to Motivation</h4><p><strong>1. Know what the business wants from your team</strong>. This is not as easy as it sounds, particularly for early managers who think “what the business wants” is just “what my manager wants”. The problem is, your manager might have the same misaligned motivation problem as you and your team. To get on top of this involves working with your Peers/Partners, and putting time into strategic Product Management, which are parts 5 and 6 of this series, so I won’t dive here. An important part is the need to get some sense of tiering of importance/value of the asks. Another early manager mistake is taking over a team which has been mismanaged, with a massive backlog of unmotivating work for very important and vocal stakeholders, and the manager feels their only option is to get their team to spend 100% of time on, even if it’s going to take years to address them all. They are then surprised 12 months in when their whole team starts quitting.</p><p><strong>2. Know your team member’s motivations and if they are being hit</strong>. The main mechanism is 1:1s, covered by the last post. What the outcome of those should be is for everyone in your team, you should have an always up to date mental tally of what they have done the last 6 months, how motivating it was, and what they want in the future. Importantly, this is an area where following the <a href="https://en.wikipedia.org/wiki/The_squeaky_wheel_gets_the_grease">squeaky wheel getting the grease</a> proverb, leads to disaster. You need to watch for the motivation of your full team, not just the outspoken ones asking for more motivating work.</p><p><strong>3. The tally applies to senior engineers.</strong> Some leeway can be given to extremely distinguished engineers who have a lot of mentoring taking up their time, and a track record of having done unmotivating work in the past. Here I am thinking people with at least 15 years who would be Senior Staff/Principal Engineer level at the FAANMG companies. But especially for those in the middle (8–15 years), I can think of nothing which demotivates junior to mid engineers more than having a supposed lead engineer who is unengaged with the day to day toil that they have to deal with.</p><p><strong>4. Recognize when people are doing the unmotivating work.</strong> Sometimes this is verbal praise, sometimes it’s a prize, for some types of work (e.g. toil from the <a href="https://medium.com/@inowland/management-area-3-operations-a547f8670582?source=friends_link&amp;sk=175d73e341380874ba3b240ad45f45db">Operations</a> section) you may want to measure and track. Failing public recognition, even private recognition (“I know this isn’t going to be fun for you for the next month, but the business really needs it”) works better than just “you need to do this as its the right thing for the business”.</p><p><strong>5. Start scheduling opportunities.</strong> My magic number is someone should get a decent amount of time to do something motivating an ideal of every 3 months, and at least every 6. While for some type of people and motivations, you may be able to duplex the motivating work with the unmotivating (e.g. if they want to go to a training course, etc), often the motivating work will need close to 100% of someone’s time. Thus, assuming the business needs things continuously delivered, you need to treat “motivating work” as something you explicitly schedule for.</p><p><strong>6. Start working on projects and their justifications.</strong> “Person X has done some great but super-unmotivating work and this project is to let them have some fun”, never works as a story to sell a project, not to your team, not to your manager, not to your peers. You need to find a match between what the person finds motivating and something that does have sufficient business value. Sometimes this is just pulling a future priority forward, at which point as long as its scope is not too big, you won’t get many questions. Usually what I end up doing is either that, or finding something that even if the expected business value is not a top priority, it has high value:</p><ul><li>In the first order as a potential outcome. I.e. “this probably won’t work, but if it does it will be game changing”.</li><li>As information. I.e. “Even if this doesn’t work out, it will answer a whole bunch of longer term strategic questions”.</li></ul><p>The main thing is finding the right project for each person is an iterative process, because as much as the subtext of needing more motivating work is there, all work your team does needs a story that your team, peers and leadership all can see that it is adding some business value.</p><p><strong>7. Time box, and beware silver bullets.</strong> From the last section, as value of a project gets more probabilistic, you need to limit potential downside. This can be done by putting hard limits of time (i.e. time boxing), cutting all fat in features, and insisting on incremental delivery. This is particularly important as there is a tendency for these types of projects to be around using/evaluating a new technology, often as part of a rewrite of an existing system (i.e. resetting the timeline in the diagram above). Beyond rewrites taking <a href="http://www.25hoursaday.com/weblog/2008/08/04/AvoidingTheSecondSystemEffectInSoftwareDevelopment.aspx">much longer than expected to capture all existing functionality</a>, you also have the problem that the new technology is only not a <a href="http://worrydream.com/refs/Brooks-NoSilverBullet.pdf">silver bullet</a> to the inherent complexity in the existing system, but also brings as many long tail negative downside as benefits. So you end up not meeting any business requirements for much longer than you expected, particularly when you do multiple new technologies at once.</p><p>Thus my rule here is any team of ~7 gets to choose one new technology a year to trial, but thats it. As a manager, that may mean you have to pick and choose between your team members most-motivating projects, finding something for the others which while still motivating, does not carry as much ongoing cost. But that is part of the art here, of picking the right motivating projects, and convincing everyone you are doing the right thing for the business.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=148244bbb0d9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Management Area 4: People, Part 1 — Understanding Motivations]]></title>
            <link>https://inowland.medium.com/management-area-4-people-part-1-understanding-motivations-35beea0a8a52?source=rss-81db999c97a7------2</link>
            <guid isPermaLink="false">https://medium.com/p/35beea0a8a52</guid>
            <category><![CDATA[management]]></category>
            <dc:creator><![CDATA[Ian Nowland]]></dc:creator>
            <pubDate>Wed, 07 Aug 2019 00:13:28 GMT</pubDate>
            <atom:updated>2019-08-23T22:14:22.286Z</atom:updated>
            <content:encoded><![CDATA[<h3>Management Area 4: People — Understanding Motivations</h3><h4>It’s all about having authentically honest 1:1s</h4><p>(Part Four, subpart one, of <a href="https://medium.com/@inowland/the-seven-areas-of-software-management-be89d213ea7?source=your_stories_page---------------------------">The Seven Areas Of Software Management</a>)</p><p><em>“If I want to say to the world ‘eating people is wrong’, and that’s what I have to say — I might as well not say it. I might as well go home and start manufacturing earth shoes, or something of this kind”</em> — Donald Barthelme, interview with Judith Sherman and Charles Raus, 1975.</p><p>I start with the Donald Barthelme quote, partly because he is my favorite author and not enough people know of him, but mostly because it speaks to the difficulty of saying something new and interesting about people management in software development. A lot interesting has been said:</p><ul><li><a href="https://hbr.org/2017/08/high-performing-teams-need-psychological-safety-heres-how-to-create-it">“Psychological Safety”</a>, by Laura Delizonna, Stanford</li><li><a href="https://www.themuse.com/advice/the-5-things-all-inclusive-leaders-do-every-single-day">“5 Things Inclusive Leaders Should be Doing”</a>, Jim Morris of Moementum Inc., a global boutique training consultancy.</li><li><a href="https://www.inc.com/justin-bariso/google-spent-a-decade-researching-what-makes-a-great-boss-they-came-up-with-these-10-things.html">“Googles 10 Things of What Makes a Great Boss”</a></li><li><a href="https://www.entrepreneur.com/article/274001">“6 Steps to Build a Strong Team”</a>, Cynthia Johnson, CEO of Bell + Ivy</li><li><a href="https://www.inc.com/peter-economy/5-ways-to-build-an-extraordinary-team-culture.html">“5 Ways to Build an Extraordinary Team Culture”</a>, Peter Economy, Management Author</li><li><a href="https://medium.com/@tsunamino/setting-goals-with-your-engineers-that-dont-completely-suck-cb76b87e4275">“Setting goals with your engineers that don’t completely suck”</a>, Danielle Leong, Engineering manager at Github</li><li><a href="https://blog.jostle.me/blog/how-to-build-a-great-team-and-culture-60-pointers/">“How to Build a Great Team and Culture? 60 Pointers”</a>, Tanmay Vora</li><li><a href="https://blog.bonus.ly/how-to-keep-engineers-happy-engaged-and-motivated">“How to Keep Engineers Happy, Engaged, and Motivated”</a>, George Dickson</li><li><a href="https://www.15five.com/blog/one-on-one-meeting-checklist/">“The Only One On One Meeting Checklist You Will Ever Need”</a>, David Hassell, Cofounder &amp; CEO of 15Five</li><li><a href="https://www.extension.harvard.edu/professional-development/blog/how-adopt-coaching-mentality-and-practice">“How to Adopt a Coaching Mentality and Practice”</a>, Madeline McNeely and Michelle Ehrenreich, Harvard Extension School.</li><li><a href="https://www.joelonsoftware.com/2006/09/07/a-field-guide-to-developers/">“A Field Guide to Developers”</a>, Joel Spolsky, Fog Creek et al.</li><li><a href="http://www.elidedbranches.com/2018/07/delegation-when-being-helpful-is.html">“Delegation: When being helpful is actually hurting”</a>, Camille Fourner, ex-CTO Rent the Runway, and ex-being my manager.</li><li><a href="https://larahogan.me/blog/coaching-reflections/">“Coaching Reflections”</a>, Lara Hogan, Management Consultant</li><li>“<a href="https://firstround.com/review/three-powerful-conversations-managers-must-have-to-develop-their-people/">Three Powerful Conversations Managers Must Have To Develop Their People”</a></li><li><a href="https://rework.withgoogle.com/subjects/managers/">Google Re:work Management Guides</a></li></ul><p>I even have my own lists, in the appendices.</p><p>These are not linked to ironically, they are all great reading. But they are linked to melancholically — all this great writing, but I think everyone would agree we don’t have enough good (let alone great) managers in software development. Which comes to my interesting point, which is I have had some really good managers over the years, and they did less than 10% of what is described above. But, what they did do was find the subset that worked to understand my motivation and for them to be interested in it.</p><p>Naming names. Camille Fournier, linked above, author of <a href="https://www.amazon.com/dp/B06XP3GJ7F">The Managers Path</a>, a guide to software management, was my manager for 18 months, and is a great manager who I would happily report to again. And yet I keenly remember her first 1:1 with me, when she told me “as a senior manager I expect you to own your own development”, and it was hard not to smirk, because throughout my career, through being a junior engineer to being a senior manager, through great managers and poor, that was always the case. All that talk about coaching, and guiding, and whatever bullshit my manager was supposed to be doing, it never happened. Outside of once or twice a year feedback session (not to be neglected, but only twice a year), I owned my growth; my managers always had other problems to spend time and focus on.</p><p>The point is though, that I just categorized some managers (i.e., Camille) as great; even though they didn’t in the first order care much about focussing on my growth. And the reason for that comes to what I believe is good people management. Which is it’s not about growth, rather it’s about motivation. And what they did is: (1) use 1:1s to understand my motivation, (2) use 1:1s to make sure I was being motivated, and (3) spent time and capital to make adjustments when that was not the case.</p><p>That’s it; that’s sufficiently good people management.</p><p>By “sufficiently” I mean, I have worked with some extremely successful managers who did nothing more, instead they put the rest of their time and focus in other areas (usually product and/or engineering). But to the extent they were great at that, and built a team motivated similarly, their team loved it; loved their sense of learning and being engaged in delivering by someone so skilled and passionate go about their biggest passion. Needless to say, that style of management is very different than someone following a step by step of “coaching great 1:1s”.</p><p>And yet there is a struggle. Which is, how many people are motivated by the things you are, and/or how long is your personal motivation clearly the right thing for the business? I.e. these types of “single motivation” teams are unstable over time, particularly as they grow. And so, there is only one goal I care about, for myself personally, and any manager under me:</p><ul><li>Goal: No people leave you team (either resigning or transferring) without you giving your manager a heads up of potential risk ahead of time.</li></ul><p>This is the only goal that matters because if you understand all your team members motivations and whether they are being fulfilled, then you will know whether they are starting to look around at other possibilities. Now to get ahead of this, I have found it useful to have a supplemental goal:</p><ul><li>Goal: 100% success in engagement survey of team that they are happy in their role and their manager, or those who are not have been escalated beforehand. The best survey I have found here <a href="https://rework.withgoogle.com/guides/managers-give-feedback-to-managers/steps/try-googles-manager-feedback-survey/">is Google’s</a>.</li></ul><p>To succeed at these goals as your team diversifies, you need to get through an inflection point of management, which is realizing that most people don’t have the same motivations as you. I.e., There is a bunch of managers who succeed early on hiring people just like them, who they motivate to do great things, but then crash and burn badly at the next phase of leadership with a more diversely motivated team. Worse, often not only do they fail to understand others’ motivations, they blame them for being different, while being blind to the fact that the team is only successful because those other motivations compliment the people more like them.</p><p>So coming back to my list of what managers need to do: (1) use 1:1s to understand my motivation, (2) use 1:1s to make sure I was being motivated. So how to do that? The first thing is to have a framework of motivation. This is mine:</p><ul><li><strong>Challenge</strong>: people who are fulfilled by conquering a challenge.</li><li><strong>Flow</strong>: people who are fulfilled by the state of <a href="https://link.springer.com/article/10.1007/s10270-017-0621-x">flow</a> that comes from thinking deeply about a problem.</li><li><strong>Impact</strong>: people who are fulfilled to see that what they have done has shown value; they love the feedback loop of “did something this week”.</li><li><strong>Autonomy</strong>: ties to the last three, but it’s people who are fulfilled to see that value has come from their choices, as opposed to just being a cog in a machine that would have happened whether they were their or not.</li><li><strong>People/Team</strong>: people who are fulfilled by the first order helping of other team members, often people-pleasers whose motivation comes from seeing that people around them have been helped by their work.</li><li><strong>Growth</strong>: people who are fulfilled only when they feel like they are being personally stressed to learn new things to succeed. This can be technical learning, but often is just any type of learning they can see the value of</li><li><strong>Recognition</strong>: similar to People/Team, except its motivation that comes from having explicit recognition, sometimes within the workplace, but sometimes it’s a peer group outside as well.</li><li><strong>Compensation</strong>: how much someone gets paid, which comes last as while sometimes the absolute dollar figure matters, usually what you find is it’s a relative value that matters to most people, which ties to the last area.</li></ul><p>Next: how to use the list to understand what motivates your team? This is where 1:1s matter; they are your major opportunity to understand each team member’s motivations, and whether they are being hit. Now, the challenge is the way to do so is an art specific to yourself and to the person. My default technique is to bring a list of 3–4 relevant items to talk about (based on other 1:1s I have had, and events), with the important thing being the discussion is about coming to an understanding of the tradeoffs, rather than making a decision. Through this I can understand two situations: why they are unmotivated to do something I think is the right thing, or just as often when they are doing something right but seem unmotivated in doing it.</p><p>While this works for me, I wouldn’t prescribe it to all. I am good at tracking items across emails/meetings, and have a dry manner which works well to talk about challenges as problems. But others don’t have that, and need to find something else. My last manager at Amazon, more empathetic than I, would ask “how happy are you with your job right now on a scale of 1–10” as his opening statement for a deeper discussion. If I did so it would come across as hokey, but it worked for him.</p><p>While the best is hard, the worst managers I have had are easy to describe: they would show up to 1:1s with nothing to say, nothing to talk about. The absolute worst would occasionally bring one thing, an occasionally rebuke, which I am sure they considered “feedback”. However, even my best managers, as do myself, occasionally struggle to be present in 1:1s when other burning things clearly had that attention. That’s fine as a one off, but I was very much tallying when it was a recurring issue.</p><p>The thing about this, and reading all those guides above, is finding a subset that consistently and authentically works for you. By “authentic” I mean you need to find techniques that for you work to understand and empathize with others, but don’t leave you feeling like you are faking. Because even if they don’t work it out, it will drain you — there is too much stress in being a successful people manager to add “I need to be a fake version of myself to succeed” to the list. Becoming a successful <em>and happy</em> manager is about finding the right mechanisms that succeed for you while you can still be yourself.</p><p>Now, you have read all those links and are having great 1:1s that work for you, but that still leaves the third point: how to spend make adjustments to motivate people when what motivated them seems misaligned to business value. That is pretty long so I am going to leave it for the next post. In the meantime, below are my checklists of people management.</p><h4>Appendix 1: My Checklist for Understanding People</h4><p>For everyone who reports to you:</p><ul><li>Are you having 1:1s with every week or at least every second week?</li><li>Do you know what motivates them?</li><li>Do you know what they think of all the members of their team? Do you understand who they think of as performing strongly or not? Do you understand what roles people are taking willingly, vs feeling like that are being forced to take?</li><li>Do you have a good understanding of performance of their performance recently, both absolute and in the eyes of their peers?</li><li>Do you know their strengths and weaknesses?</li><li>Do you know where they think they want to take their career long term?</li><li>Do you understand whether team dynamics, including possibly your role on the team (e.g. taking away a leadership opportunity doing something you enjoy too much), are holding someone’s motivations back?</li><li>On all the above, particularly for junior team members, can you do them while dealing with the fact that since they see management as an “authority” rather than a “peer who helps them”, many of them will not be honest on their initial answer.</li><li>For low performers, have you asked yourself honestly “should I be deliberately managing this person up or out”, or are you just stringing along doing the grind</li></ul><h4>Appendix 2: Checklist for When You Start in a New Role</h4><p>Know your people through a first 1:1&#39;s:</p><ul><li>What do they own?</li><li>What is their background?</li><li>How do they spend their time? (Useful categories may be manual ops/leadership/administrative features/re-engineering)</li><li>How do they want to spend their time?</li><li>1:1 topics for senior engineers, or managers under you</li><li>Preferred Communication (email vs chat vs verbal, interrupt driven vs rollups)</li><li>Preferred 1:1 mechanism: frequency, what we like to cover?</li><li>How each likes to give and receive critical feedback and surprises?</li></ul><p>Establish your processes:</p><ul><li>How often should I have 1:1 with my directs?</li><li>If I have them, how often should I do skip 1:1s?</li><li>How often should we meet as a team? What do we want to talk about? What should we cover? Examples are: ops review, project reviews, team presentations.</li><li>How often should the team have offsites and social activities? Together? As separate teams?</li></ul><p>Understand company’s processes for your people:</p><ul><li>What is the job families for engineering?</li><li>What are the job levels for engineering?</li><li>Is there a feedback/review process?</li><li>How does calibration happen?</li><li>How do promotions happen?</li><li>How does this all feed into compensation review process?</li><li>Managing Out: how does it work (i.e. timing, documentation requirements)?</li><li>Timeoff: What do people get? How to report/review? What about FMLA?</li><li>Terminating: (resignation or surprise) What happens? Who do I engage?</li><li>How much conference attendance is acceptable at team level, and individual level?</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=35beea0a8a52" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Management Area 3: Operations]]></title>
            <link>https://inowland.medium.com/management-area-3-operations-a547f8670582?source=rss-81db999c97a7------2</link>
            <guid isPermaLink="false">https://medium.com/p/a547f8670582</guid>
            <category><![CDATA[development]]></category>
            <category><![CDATA[management]]></category>
            <category><![CDATA[sre]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Ian Nowland]]></dc:creator>
            <pubDate>Sun, 12 May 2019 18:11:02 GMT</pubDate>
            <atom:updated>2019-05-14T03:50:47.478Z</atom:updated>
            <content:encoded><![CDATA[<h4>Is the built thing going to keep running?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*boxKl-phYpkOaOMEAH8QrA.png" /></figure><p>(Part Three of <a href="https://medium.com/@inowland/the-seven-areas-of-software-management-be89d213ea7?source=your_stories_page---------------------------">The Seven Areas Of Software Management</a>)</p><h4>Overview</h4><p>When I entered the industry at the turn of the millennium, it was for a company doing “boxed software” drops twice a year, for which there was little operational burden — staffing a “tier 3 support” email rotation to triage field issues, and the occasional ask to stop feature development and do a hot fix for a customer issue. Thus when I journeyed across the world to Seattle to join Amazon.com in 2006, I was surprised to be told “here’s your pager and you will be on-call 24/7 every 6 weeks”.</p><p>In the time since, much of the software industry has moved to this model — coupling agile planning, software as a service, continuous delivery, and development teams owning operations. These embrace that since software has no physical aspect, you can succeed in building a great product by shipping something minimal quickly and then continuously improving as you learn exactly where it is needed. The challenge is modifications take time to develop, so if you ship something under-specified to how customers start using it, you can find yourself in operational hell — your development team spending all their time doing hands-on toil keeping the system running, with no time to develop the reliability improvements to stop needing to do so. Preventing this from happening is especially difficult as shipping new features to win customers is far more visible to the organization than the work to ensure the system will reliably operate after they are won.</p><p>As a result, a number of practices have come about to balance features with reliability, with <a href="https://en.wikipedia.org/wiki/DevOps">devops</a> and Site Reliability Engineering (<a href="https://en.wikipedia.org/wiki/Site_Reliability_Engineering">SRE</a>) being predominant. They are well worth reading about, although I find SRE has muddied the waters blurring questions of organizational structure into questions of practices, and while org structure is important, it doesn’t help where most development managers find themselves today:</p><ul><li>Your team develops software</li><li>Your team owns tier 1 support for all or most issues</li><li>Your team is expected to monitor the deployed software and fix issues ahead of customer impact</li><li>Your team is software engineers none of whom are inherently motivated to do most of this.</li><li>The business wants you to ship features as fast as you can, and expects you personally to do the balancing with reliability to make sure that it doesn’t come crashing down.</li></ul><h4>Understanding Your Position</h4><p>A lot like Engineering, most of tactics in Operations is process. A list of basics are:</p><ul><li>Telemetry and Production Debugging —e.g., logs, metrics and APM</li><li>Synthetic/Active Monitoring</li><li>Capacity Planning and Scaling</li><li>Change Management and Deployments — e.g., integration testing, phased rollouts, feature flags, service discovery, canaries</li><li>Post mortems</li></ul><p>Again the questions are:</p><ul><li>What is being done by your teams?</li><li>What are organization policies?</li><li>What are industry best practices?</li><li>What is your engineer engagement and happiness with processes?</li><li>How is your team’s performance perceived by partners and customers?</li></ul><p>Whether you are new to a team, or have a team you have been with for a long time, one things to be wary of is <a href="https://danluu.com/wat/">normalization of deviance</a> — practices that are substandard to an outsider, but inside the team are seen as acceptable as “it hasn’t caused problems yet”. In Operations, this is often only because of a reliance on the deep knowledge of the experts who built it also being the ones who are operating it, which leaves you setup to be in a bad place when they leave:</p><ol><li>System quickly built to handle an acute business or operational issue, with almost no investment in operational best practices. Starts showing success, everyone celebrates, gets promoted, etc</li><li>System starts having operational issues, but the deep experts who built it know just where to quickly place the duct tape for any specific issue, so operational load seems fine.</li><li>System scale of usage and breadth of use cases keeps growing</li><li>Toil load has scaled linearly and is now in a bad place. Deep experts burn out doing too much toil and leave for greener pastures</li><li>Team brings in new engineers, they make mistakes in placing the duct tape, now customers and the business start feeling impact.</li><li>Business finally realizes the team is in a bad place</li><li>System won’t stop growing, so team is now in such a place it takes a couple of years to get on top of the reliability work that should have been done back at step 1, costing the business growth opportunities, the team its good and deep engineers, and you personally a lot of grey hair.</li></ol><p>So when you ask these questions about process, make sure you are really understanding risk. For teams in a poor place tactically, these are the types of goals to take:</p><ul><li>Goal: Establish a on-call rotation across your team</li><li>Goal: Have your team deliver a well received operability review for X</li><li>Goal: All systems have a CI pipeline</li><li>Goal: All systems have a canary deployment mechanism</li><li>Goal: All systems have basic active/synthetic monitoring</li><li>Goal: All post mortems reviewed within N days</li><li>Goal: All post mortem action items closed out within N days</li></ul><h4>Getting to a Strategy</h4><p>My favorite question when I interview a manager who owns 24x7 operations is “how do you sleep at night?” Beyond the obvious, this is trying to get at how they handle this area strategically — which especially under pressure to ship features, it’s difficult to rely on “because my processes are good” as that is a static sample, whereas the tradeoff of reliability vs features is ongoing, and so you need a way of getting the balance right to get ahead of future issues. To which I have found the best practices is 5 mechanisms/metrics:</p><ul><li>Pager Load</li><li>Toil Load</li><li>Risk Backlog</li><li>Availability and Performance SLOs</li><li>Weekly Operational Reviews</li></ul><h4>Pager Load</h4><p>The purpose of paging a developer on-call is when you have an urgent operational issue you need a developer to resolve. Thus it maps well to customer pain. But of course a developer being paged is a pain, even in working hours it distracts from what they were focussed on. And on the weekends, or at 2AM in the morning, it is much worse.</p><p>The nice thing about pager load is it’s easily measured. The difficult thing is not all pages are the same amount of pain— an event causing 7 pages is not the same as 7 events, and 7 customer impacting events over 7 nights at 2AM are not the same as 7 false alarms during office hours you were warned about. Still, having argued for mechanisms that account for those in the past, I now find it is simplest to ignore special cases, just focus on the raw count per week, and have a high bar. Which for a 24x7 ~6 person development team rotation is:</p><ul><li>&lt; 2 a week — team is healthy, no work needed</li><li>Between 2 and 5 a week — team is healthy but walking the edge, should be spending some development time removing causes of pages to stop growth of count</li><li>&gt; 5 a week — Team is unhealthy. Resources need to be moved off features to get load back under 5.</li></ul><p>These come from my last VP at Amazon who worked with HR to correlate pager load with attrition rates, and maps well to my seven years of experience being a primary on various pager rotations. Now as they pollute your signal, this simple measure means you will need to invest work in “deduplicating” pages and/or eliminating “it only fires when we deploy so its okay” type pages, but to me that is worth the cost vs a more complicated metric that can’t be compared across teams. So with that in mind, the goal to take is:</p><ul><li>Goal: Average less than N page events a week (N is 5 when you are taking the goal and the team is in a bad state, N is 2 when the team is in an okay state).</li></ul><h4>Toil Load</h4><p>The concept of <a href="https://landing.google.com/sre/sre-book/chapters/eliminating-toil/">toil</a> in devops owes its definition and popularization to Google SRE. Toil is not inherently impactful to customers, but is painful to most engineers, for the reason that most people who become software engineers do so as their job realizes two motivations:</p><ul><li>Creating software that improves the world (i.e. building)</li><li>The sense of <a href="https://link.springer.com/article/10.1007/s10270-017-0621-x">flow</a> that comes from contiguous time writing challenging code (i.e. coding)</li></ul><p>Toil gives no such motivation. Engineers will do it because it’s the right thing for their team, or the right thing for the business and they want to get paid. But that is a more basic level of motivation than what other roles usually offer, and so high toil causes high turnover, which in turn causes loss of knowledge, and so operational errors, and so customer pain.</p><p>So how to measure toil? For lightly operationally loaded teams, the easiest option is to have a policy that the only person who touches production (even for deployments) be your on-call, at which point you can proxy toil with that. That works well for about a third of teams. If the load is too high for that, then having two rotations — a support on-call and a deployment on-call can be enough. That works well for about another third. In both cases the nice thing about this state is everyone not on-call should be able to ignore the production system, and focus on engineering.</p><p>Where teams struggle with those approaches is when they have high load and are “siloed” — most team members can only go deep on a subset of systems. So when you have an operational issue, you need engagement of people who aren’t on-call. That is a bad state for their flow and something you need to resolve, but it takes time to build tooling and do cross-skilling. So in that situation, you need to use your Execution mechanism to estimate toil. If the toil work is mostly non-urgent then you can integrate it into your Scrum process and use the estimates from that. If the toil is more urgent, you probably need to move to Kanban, which while non-ideal for software development, is a good way of handling a high-interrupt reality. In either case I have found a line manager can build a pretty good estimate of toil load just through that process.</p><p>One area to cover with toil is: what is the right amount? While some would say zero, the reality is systems are complex, support cases happen, and automation isn’t free. Google documents its SRE teams goal is 50%, but that is too high for a team doing both features and reliability; rather you want to target about 25% — in a six person team just the on-call fully focussed on support and half a head doing things like deployments or region builds. With that in mind, potential goals are:</p><ul><li>Goal: Establish measure for toil. Drive toil down to 25% total time by EOY.</li><li>Goal: Establish processes and tools so that the on-call is able to fully handle 90% of high and low priority issues without escalating to other team members</li><li>Goal: Average less than N low priority system monitors issues a week</li><li>Goal: Deliver runbooks and tools to support team so they are able to escalate less than N customer support issues to your team a week.</li></ul><h4>Risk Backlog</h4><p>A lot of people proxy toil for all technical debt, but that misses the risk that has built up in systems under ever increasing scale which is not causing issues today but could cause very impactful issues in the future. So just focussing on toil means you spend too much time urgently band-aiding as things break, where you should be re-engineering the system to get ahead of the next multiple of scale. In EC2 VPC when we measured we found our engineers time divided equally between toil, automation to reduce toil, re-engineering systems to scale and get ahead of systemic risk, and delivering new features.</p><p>Now the first question is how to get to a list of risks? I have found that part isn’t hard; as much as normalized deviance is real, if you asks your engineers “where is all the risky parts of our system”, they will have no trouble answering, in fact the opposite. Actually that is where the normalized deviance comes from, the list looks so long it’s easy to give up hope of impactful change. And so your challenge is to come up with the subset to take action on. Here it turns out there’s a whole industry of IT risk management that has built frameworks; I particularly like the Impact and Likelihood matrix explained <a href="https://en.wikipedia.org/wiki/IT_risk">here</a>. The main thing I add is a “level of effort” column, as you will find varying amounts of effort to handle items at the same severity level, and so it follows you use effort to guide prioritization. Another thing is to include both security and operational risks; indeed this is your best way of normalizing the importance of the two. With all that in mind, the goal is:</p><ul><li>Goal: Build list of the 10 highest severity operational and security risks, with a funded plan to address in the next 12 months.</li></ul><h4>Availability and Performance SLOs</h4><p>One concern as you move towards automation and limits to get on top of operational load, is you have made it hard to understand what customers require from your system — i.e. you have broken the “build as you learn” model. That is a fair but misplaced criticism — yes engineers getting woken up because your system is breaking is feedback, but that doesn’t mean to say its efficient feedback. <a href="https://landing.google.com/sre/workbook/chapters/implementing-slos/">Service Level Objectives (SLOs)</a> and Error Budgets are formalizations from Google SRE that address this. Now this is one where I think you need to be careful about their rhetoric, as it relies heavily on the notion that for these to be effective, you need a structure of a SRE org to hold a Feature organization accountable for their SLOs. Whereas many of us experienced in other org structures know that is not the case at all.</p><p>The idea of metric driven organizations is old. The challenge is making metics have teeth in management prioritization decisions. Now a naive engineer will say “management needs to work out what they want”. Whereas an engineer-turned-manager knows “management” is 10+ different stakeholders with situationally distinct authority and influence, all with different qualitative opinions on what is right for the business. What a good metric does is capture business impact in a way all those stakeholders agree is reasonable, and thus be reasonably bound by. For that, there is a few best practices:</p><ul><li>Fewer SLOs is better. If your stakeholders have to drop something they want every time one of your ten SLO’s goes red, they are going to question why all ten matter. Ideally you have just one for availability, and one for performance. Now internally, you should have more, particularly around tracking deployment regressions. Just don’t expect those to have teeth with stakeholders.</li><li>Ideal SLOs capture all support cases where your system is root cause. I.e., When you have upset customers and argue “but we met our SLO”, even just internally, you have weakened the authority of your SLO.</li><li>Simplicity is ideal, but not authoritative. Simplicity aids ease of understanding across a broad audience. But if simplicity also means you are not capturing customer pain, then you should be fine going for a more complex SLO.</li><li>Durability is key. If you ask a stakeholder to sign off on a tradeoff based on a SLO today, and 6 months later you say that SLO doesn’t matter, then you have weakened your authority in having SLOs be binding. The best SLOs are ones that last for years, being slowly “ratcheted up” in compliance rate.</li></ul><p>An important part of SLO’s effectiveness is you need to get on top of internal violations before asking for external tradeoffs to be made. That is, if you are behind on your availability SLO, for example because your engineers didn’t sufficiently test for regressions, don’t expect you can ask to drop a feature so you can do a rewrite. This is especially important when an engineering team thinks their only way out of operational hell is a total rewrite, and they want their management to push back on all stakeholder asks on the existing system to make that happen; including reliability asks. While this may seem reasonable in terms of optimizing resources, from a stakeholders perspective, why should they trust you to have an all new system operating well, when you demonstrably are failing at doing that with the current house?</p><p>As teams behind a feature grow, SLOs become the most powerful way to keep an organization focussed on customer experience. In my experience Amazon was happy to devote ~10% of engineering headcount to building systems to measure SLOs. A particular callout was <a href="https://www.linkedin.com/in/davidrr/">David Richardson</a>, the best manager I ever worked with, who took over EBS after <a href="https://aws.amazon.com/message/65648/">their outage in 2011</a>, and whose main mechanism for balancing reliability vs features was a non-trivial performance SLO backed by a ~10 person team of engineers and data scientists who measured compliance both positively and synthetically. That may seem hard to justify the resources, but unless you are running a stateless request-reply service where you can just use nines of availability and response time, you need to invest in good SLOs for them to have an impact. So with that in mind, suggested goals are:</p><ul><li>Goal: Establish an availability SLO and hit it.</li><li>Goal: Establish a performance SLOs and hit it.</li><li>Goal: Ratchet up your current SLOs and hit them.</li></ul><h4>Weekly Operational Reviews</h4><p>Everything until now has been about how to establish good goals, but has not addressed much of the real challenge: how to achieve the goals — i.e., how to make the right calls on reliability vs features, including getting buy in from your stakeholders to let you act on those calls?</p><p>This is best described through anecdote. <a href="https://www.linkedin.com/in/andy-jassy-8b1615/">Andy Jassy</a>, CEO of AWS used to get on stage at every quarterly AWS all-hands and open with “Operations is our number one priority”. Now if you observed Andy from afar, this seemed like hypocrisy. Andy’s at heart a product guy; what excited him was new products, new features and new business. And yet, when EC2 VPC started struggling under scaling in 2014, Andy asked for a monthly review with EC2 VPC’s management, with us writing a <a href="https://medium.com/@inowland/using-6-page-and-2-page-documents-to-make-organizational-decisions-3216badde909?source=friends_link&amp;sk=a84453eebf33bd437ca4b96975131a18">6 pager</a> to cover both our performance against our operational goals, and short and long term plans to resolve. And Andy did not just attend the meeting — he was deeply engaged, with non-trivial questions every meeting, and willing to make tradeoff calls on the spot if need be (we would sometimes use the opportunity to pose such questions). And so this was a key lesson in management leadership — Andy spending his time and focus holding us accountable for our performance was all that was needed to empower us to make the right calls. I have a saying for this — “People follow their leaders focus”.</p><p>The simplest mechanism in this space is a weekly operations review, covering performance against goals, post mortems, and action items. That was standard <a href="https://aws.amazon.com/blogs/opensource/the-wheel/">AWS practice</a>, indeed <a href="https://www.linkedin.com/in/charlie-bell-05446835/">Charlie Bell</a>, head of AWS Engineering, turned up to a 2 hour one and was engaged and asked deep questions. I will be honest though, my thinking at the time was that these tended toward a combination of bureaucracy and bad theatre — too many people playing up their “chops” to the senior audience. So these are the best practices I came to:</p><ul><li>10 people ideally, 20 people at most.</li><li>As opposed to bureaucratically following an agenda based on goal compliance, content should be week to week tailored to what is pertinent, actionable, and broadly interesting to all attendees. (This likely means you need to ask someone to spend 4–8 hours a week to prepare an agenda for each meeting).</li><li>Senior leadership who own/influence reliability vs features calls must attend and actively engage in a way that engineers can see they understand they are engaged and so accountable.</li><li>Meetings should telescope up the org, with more senior leadership and a smaller emphasis on deep details vs a heavier emphasis on broad lessons and trade offs as it moves up.</li><li>Ideally 30 minutes.</li></ul><p>And with that the goals for any manager are:</p><ul><li>Goal: Establish a weekly operational review meeting.</li><li>Goal: Score 80% on agreement on “Our management understands our operational challenges” in a yearly survey of engineering.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a547f8670582" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Management Area 2: Execution]]></title>
            <link>https://inowland.medium.com/management-area-2-execution-c19192030ff4?source=rss-81db999c97a7------2</link>
            <guid isPermaLink="false">https://medium.com/p/c19192030ff4</guid>
            <category><![CDATA[engineering]]></category>
            <category><![CDATA[project-management]]></category>
            <category><![CDATA[process]]></category>
            <category><![CDATA[managment]]></category>
            <category><![CDATA[agile]]></category>
            <dc:creator><![CDATA[Ian Nowland]]></dc:creator>
            <pubDate>Mon, 29 Apr 2019 00:21:16 GMT</pubDate>
            <atom:updated>2019-11-18T00:46:24.612Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qmC1dhOEMw_4awMaT-Uf8w.jpeg" /></figure><p>(Part Two of <a href="https://medium.com/@inowland/the-seven-areas-of-software-management-be89d213ea7?source=your_stories_page---------------------------">The Seven Areas Of Software Management</a>)</p><h4>Overview</h4><p>This is the second easiest area to get on top for new managers, as it’s usually one of the reasons they are made a manager: the ability to get groups of people to coordinate to execute and deliver. There is two aspects of managing execution:</p><ul><li>Project Management: managing individuals to deliver a single goal.</li><li>Process Management: managing what your team members are working on as they are being asked to work on multiple projects/goals.</li></ul><p>You project manage to deliver a goal. You process manage to optimize the simultaneous delivery of multiple goals .</p><h4>Understanding Your Position</h4><p>To get on top of your state for each:</p><p>Process:</p><ul><li>Do you understand the full list of projects and other goals your team needs to deliver?</li><li>Do you understand their relative priorities?</li><li>Are you overcommitted?</li><li>How well is your team executing on the commitment?</li><li>Are people inside the team happy with your state?</li><li>Are your partners, management and customers happy with your delivery rate and schedule?</li></ul><p>Projects:</p><ul><li>For each project your team is being asked to do, is there a clear project owner? If not, are people assuming you are the owner?</li><li>What are the deadlines that your team needs to hit to not be the long pole?</li><li>For those projects you own: What buffer do you have? What risks are there that will eat that buffer? Do your partners, management and customer understand the risk?</li></ul><p>The thing here is not just understanding state, but also understanding perception of state. A large amount of the the tension in organizations is disagreements on the absolute priorities of projects. However this gets stated by stakeholders frustration at execution rate on their favored project. Thus, succeeding through strategy is developing mechanisms to enable the communication of priorities to happen.</p><h4>Getting on top of Strategy</h4><p>Both project and process management have a great deal written about various techniques which I won’t recount. What I will say is they are optimizing differently across:</p><ul><li>Prioritization: Ensuring teams are working on the most important things</li><li>Throughput: Optimizing the delivery pipeline; minimizing throw-away and duplicated work</li><li>Risk: minimizing variability on quality and time of delivery</li><li>Agility: Delivering high-value work earlier to get feedback to allow course correction. This allows risk taking innovation while <a href="http://www.porticos.net/how-to-escape-from-a-rabbit-hole-problem-solving-in-product-design/">avoiding rabbit holes</a>.</li><li>Communication: Maximizing information and minimizing overhead in coordination and collaboration of everyone doing the work</li><li>Commitment: solving collective action problems across partners on implementation tradeoffs impacting them differently</li><li>Visibility: Providing management with visibility into delivery state, and so allowing escalation to resolve roadblocks and other challenges</li><li>Feedback: Creating information to improve quality of future execution</li></ul><p>There is no set of tradeoff right for all teams for all circumstances, which means there is no single “right” project or process management technique. Instead you need to be continually evaluating if you are optimal at this point in time, with a focus on understanding if all stakeholders agree. The biggest problems in execution are not usually a team not working hard on important things, rather it’s lack of communication about the tradeoffs in priorities on what the team chooses to work on, which causes wasted resources due to different teams working to different priorities, and mismatched expectations.</p><p>This can cause stakeholders to create narratives about why your team isn’t delivering the “right” things. This can cause a lot of negativity in relationships, which you want to blame on others selfish expectations of their priorities being the “right” priorities. However, unless you are spending the time communicating to help the organization understand the tradeoffs in all priorities, what else should you expect? I call this communication “owning the narrative”, and what I mean by it is:</p><ul><li>You need to have a strategy</li><li>You need to communicate that strategy</li><li>You need to be seen to deliver on your strategy</li></ul><p>Thus proposed goals to take are:</p><ul><li>Goal: Improve team’s process by implementing X (e.g. implement scrum, Kanban, etc)</li><li>Goal: Creating a mechanism to measure and communicate teams ratio of time spent between operations, tech debt reduction, and features, in a way that informs planning and stakeholders</li><li>Goal: Deliver 80% of quarterly OKRs. For all OKRs not hit, they should be escalated as stretch/miss at least a two weeks before due date.</li><li>Goal: Reduce backlog to X items by Y, ensure it never goes above X for the rest of the year</li><li>Goal: Own project management to deliver very important cross-team OKR X on time by date Y</li></ul><h4>Saying No With Data, And Delegating With Supervision</h4><p>Two technique for success here are “saying no with data”, and “delegating with supervision”. The reason is for a line manager of a team in a fast moving organization of &gt; 150 (see <a href="https://www.qualitydigest.com/inside/quality-insider-article/rule-150.html">rule of 150</a>), after subtracting the minimal time to execute tactically in the other 6 areas, the remaining time can easily be fully consumed by this area (i.e. including all the project management and communication). Now if you have a small team with a highly important goal, this can be the right thing for yourself, your team and the company. The challenge is when you start growing your team size, above about 6. Then you need to find more time and focus to be strategic in the other areas.</p><p>What is “saying no with data”? My old line in EC2 was “the two wrong answers when you are overloaded and asked to do an additional thing are yes and no.” Just saying “no” means you will be seen an obstructionist, so you need to ask for the priority. But what you find then is whoever is coming to you is almost always unaware of global prioritization, even when they assure you otherwise:</p><ul><li>“our VP really wants this” (but VPs always want many things)</li><li>“this is my orgs number 1 priority” (but your orgs priorities aren’t necessarily the same)</li><li>“this is our common VPs number 1 priority” (here you learn some VPs have many number 1 priorities).</li></ul><p>This can include even when it is your manager asking you; they often have a lot less context on your roadmaps priorities and tradeoffs than you do. Now in the moment, as long as you are aware of not assuming what someone says is what is true, then with a decent amount of time and effort you can resolve the ambiguity. But then a different stakeholder will come along with the next ones. And thus being strategic is spending the time and focus ahead of time, gathering the data in a manner you can reuse when these questions come up. Thus suggested goals are:</p><ul><li>Goal: Deliver a scoped 12 month roadmap of currently known executables</li></ul><p>With this, seeing the impact of an additional ask is quickly visible to yourself, whoever is asking, and whoever you need to escalate to get a tradeoff decision.</p><p>Delegating is for things being asked of your time and focus; finding a way to say yes by giving some of that to someone in your team. Delegating with supervision is an additional notion that comes from Andy Grove with the key aspect that you have some aspect of supervision to help the delegate succeed. There is a bunch of writing on how to delegate. Amazon treated it a very specific way, which was to talk about mechanisms. This meant creating a “mechanic” system whereby the delegate, with your supervision, is set up to succeed. This means 5 things:</p><ul><li>A goal</li><li>Identification of the stakeholders who need to cooperate for the goal to be reached</li><li>Recognition of a clear owner (i.e. the delegate), the important thing is all stakeholders recognizing their ownership</li><li>A criteria for which success/failure can be ongoingly judged</li><li>A periodic or edge triggered communication agreement for all stakeholders that facilitates coordination towards goal.</li></ul><p>Project management is one of the easiest mechanisms to set up: a project manager owns a feature with a deadline date, defines milestones as ongoing measurement criteria, hosts a status update meeting for all stakeholders after each milestone. To start getting on top of Execution, you need to start asking your engineers to own some tactical aspects. This is sometimes hard because the only people who seem to like project management are project managers, but project managers are usually not worth their overhead unless a goal is particularly big (nominally &gt;2 dev years, &gt;4 teams). But asking people to do the right thing to grow is part of being a manager. And so the type of goal to take is:</p><ul><li>Goal: Coach lead engineer X to own project management of execution of cross-team OKR X</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c19192030ff4" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>