Absolutely No Machete Juggling Rod Hilton&#039;s rants about stuff he cares about way too much. http://www.nomachetejuggling.com Software Engineering Guiding Principles - Part 2 <p>Here are five more Guiding Principles I use when making technical decisions as a software engineer. You can also check out <a href="http://www.nomachetejuggling.com/2016/06/15/guidingprinciples-part1/">Part 1</a>.</p> <p>Just as before, this list is really a list of principles I use when making difficult technical decisions or mantras I use to snap myself out of being stuck - it&#39;s really not about just how I try to write good code (SOLID, DRY, etc) although there is a little bit of that as well.</p> <h1>Perfect is the Enemy of Good</h1> <p>When it comes to designing code, I think it&#39;s better to get started as soon as possible and make changes and modifications via refactoring as needed. It&#39;s better to get something up and working quickly, rather than spending time debating in front of whiteboards about the correct way to do things. In my experience, engineers in particular have such an affinity for elegance that we can get wrapped around the axle trying to figure out the perfect, most elegant solution.</p> <p>I&#39;m not saying to write shitty code, obviously. It&#39;s still important to follow good design principles like <a href="https://en.wikipedia.org/wiki/SOLID_(object-oriented_design)">SOLID</a>, the <a href="https://en.wikipedia.org/wiki/Law_of_Demeter">Law of Demeter</a>, <a href="https://en.wikipedia.org/wiki/KISS_principle">KISS</a>, <a href="https://en.wikipedia.org/wiki/Defensive_programming">defensive programming</a>, <a href="https://christiantietze.de/posts/2015/09/clean-code/">CLEAN</a>, <a href="https://en.wikipedia.org/wiki/Separation_of_concerns">separation of concerns</a> and so on. It&#39;s just that you don&#39;t have to get every little thing perfect, it&#39;s better to get something that&#39;s imperfect but works built and then refactor to perfection later.</p> <p>Remember <a href="https://en.wikipedia.org/wiki/John_Gall_(author)">Gall&#39;s Law</a>:</p> <blockquote> <p>A complex system that works is invariably found to have evolved from a simple system that worked.</p> </blockquote> <p>It&#39;s important to realize when you or your team have gotten into a state of <strong>analysis paralysis</strong>, which is one of the reasons I like Pair Programming so much - it&#39;s handy to have a second person around to recognize when you&#39;re wrapped up analyzing instead of building. Nobody really asked you to build the world&#39;s greatest, most reusable, most well-designed system on the planet. <strong>The company doesn&#39;t need the perfect solution, it just needs one that&#39;s good enough</strong>.</p> <p>There are lots of ways engineers can get gridlocked doing analysis, and it&#39;s important to recognize all of them.</p> <h2>Premature Optimization</h2> <p>Don Knuth calls Premature Optimization the <a href="http://c2.com/cgi/wiki?PrematureOptimization">root of all evil</a>. It can happen both in code/design, as well as architecture.</p> <p>If you find yourselves talking about caching layers, circuit breakers, or geo redundancy before building even the first version of the software, you might be getting ahead of yourself. <strong>Those things are all just as easy to add later as they are to add now</strong>, so there&#39;s no reason to get wrapped up on these concerns early.</p> <p>Obviously I&#39;m not advocating writing inefficient algorithms when an efficient one is just as easy to implement, but if the code is substantially cleaner with something less efficient, leave well enough alone and just get it working. Even dumbass bubble sort is usually good enough, and it has the advantage that you remember how it works right now without double checking anything on Wikipedia. </p> <h2>Bike Shedding</h2> <p>Otherwise known as the <a href="https://en.wikipedia.org/wiki/Law_of_triviality">Law of Triviality</a>, this is when a disproportionate amount of weight is given to trivial concerns when designing something. The term comes from the fact that teams will tend to focus on the minor issues that are easy to understand, such as what color to paint the staff bike shed.</p> <table class='image alignright' width='306'><tr><td><img src="http://www.nomachetejuggling.com/assets/sheldon-cooper.png" width='300' height='187'/><p class='caption'>You're in my spot</p></td></tr></table> <p>The more time you devote to making a decision, the more you need to periodically ask yourself &quot;does this really matter?&quot; A lot of times, it doesn&#39;t matter to anyone else on the team, it doesn&#39;t matter to your users, and it certainly doesn&#39;t matter to the company. <strong>If it only matters to you, you&#39;re probably being, you know, kind of a dork</strong>.</p> <p>An entire team can bike shed as well. Recognize when your team is bike shedding and stop the conversation, drive it toward the things that matter. If people keep gravitating toward the trivial, it means that there&#39;s a lack of comprehension of the difficult decisions that actually matter. You either need to stop and get everyone on the same page about the challenging stuff, or you have the wrong group of people making the decision.</p> <h2>Overengineering</h2> <p>Premature reusability. Engineers have a tendency to want to design components to be as generic and reusable as possible, there&#39;s an old joke from Nathaniel Borenstein I&#39;m fond of:</p> <blockquote> <p>No ethically-trained software engineer would ever consent to write a DestroyBaghdad procedure. </p> <p>Basic professional ethics would instead require him to write a DestroyCity procedure, to which Baghdad could be given as a parameter.</p> </blockquote> <p>A really great example of over-engineering is found in Bob Martin&#39;s <a href="https://smile.amazon.com/Software-Development-Principles-Patterns-Practices/dp/0135974445?sa-no-redirect=1">Agile Software Development</a>. In it, Bob Martin and Bob Ross sit down to do the <a href="http://butunclebob.com/ArticleS.UncleBob.TheBowlingGameKata">Bowling Game Kata</a>, a programming exercise where you simply write code to calculate the scores for a bowling game.</p> <p>The two engineers started talking about what classes they were going to have. There would need to be a <code>Game</code>, which of course would have 10 <code>Frame</code> instances, each of which would have between 1 and 3 <code>Throw</code> instances. This seemed natural, like how you might answer a &quot;design a the object model for a bowling game&quot; question in an interview.</p> <p>But as they tried to write tests to drive out the behavior of <code>Frame</code> and <code>Throw</code> they found that there were no behaviors to those classes. A <code>Throw</code> is really just an <code>int</code>. In the end, they wound up with a simple <code>Game</code> class and nothing else, with a handful of methods on it to say how many pins were hit, and a method to get the score.</p> <table class='image aligncenter'><tr><td><a href='http://xkcd.com/974/'><img src='http://imgs.xkcd.com/comics/the_general_problem.png'/></a></td></tr></table> <p>Don&#39;t start any large endeavor with a mind on generality and reuse. Follow the <a href="https://blog.codinghorror.com/rule-of-three/">Rule of Three</a> - make everything designed for single use and naturally you will eventually discover reusable components falling out when refactoring after you&#39;ve done the same or similar things in multiple places.</p> <h2>Exception - Architecture</h2> <p>It&#39;s important to note that there is one exception to this idea: your code <em>design</em> can be just good enough. But your system <em>architecture</em> needs to basically be perfect from the start. This can be extremely difficult to get right, but it&#39;s important, so a little bit of analysis paralysis is somewhat forgivable.</p> <p>When it comes to code design, evolutionary design is the way to go - just build it and evolve it. But for architecture, get the team into a room with a whiteboard and hash out the details before you start building. <strong>Evolutionary design, up-front architecture</strong>.</p> <p>How do you know the difference between design and architecture? One analogy I&#39;m fond of is that architecture is strategy while design is tactics. Doing the right thing vs doing things right. That&#39;s a helpful distinction but I find myself most fond of <a href="http://www.ibm.com/developerworks/library/j-eaed10/">Martin Fowler&#39;s definition</a>:</p> <blockquote> <p>Architecture is the stuff that&#39;s hard to change later. And there should be as little of that stuff as possible.</p> </blockquote> <p>Anything that would be extremely difficult to change later on is something deserving of a substantial amount of upfront analysis. The language you choose for your code is architecture because changing it would require a full rewrite. If you&#39;re using a highly opinionated framework like Rails, Grails, or something that spreads throughout your entire codebase like Spring, that&#39;s architecture. </p> <p>If you go with microservices, lots of decisions that are typically architecture suddenly become design, because you could swap one microservice for another easily, or quickly swap out the language or framework of one service. However, now the contracts between services - which would be easy to refactor if they were all in a single codebase together as simple classes - cease being design and become architecture. And of course the decision to use microservices or not altogether <em>is</em> architecture. </p> <table class='image alignleft' width='306'><tr><td><img src="http://www.nomachetejuggling.com/assets/architecture.jpg" width='300' height='188'/></td></tr></table> <p>The data store you use is likely architecture. It can sometimes be easy to swap out MySQL for Oracle if you&#39;re using a strong database abstraction layer or relying on JPA or ActiveRecord something similar, but as your data needs grow you&#39;ll quickly find yourself using customized queries or perhaps even stored procedures, and migrating becomes difficult. Even if you choose something like Postgres and try to keep the option open to switch to Oracle or MariaDB, you&#39;re still picking a relational database at all, and switching to a NoSQL store would be extremely difficult so no matter how you slice it, it&#39;s architecture.</p> <p>Public-facing APIs are a strange middle ground. Once you&#39;ve decided on the APIs, they&#39;re impossible to change without affecting your users, so they&#39;re architecture. However, you can introduce a new API version later fairly easily, so it&#39;s not that hard to change your mind, making it sort of design? Of course, the WAY you version the APIs in general is architecture, because if you provide no facility for versioning early on it becomes difficult to add a new version later.</p> <p>Overall the dichotomy is subjective so you need to use your best judgement, but what&#39;s important is that you don&#39;t spin your wheels making something perfect that could be perfected later if it can be good enough now.</p> <h1>If You Break My Code, It&#39;s My Fault</h1> <p>I&#39;ve blogged about this one before, under the more provocative title &quot;<a href="http://www.nomachetejuggling.com/2011/10/21/i-broke-your-code-and-its-your-fault/">I Broke Your Code, and It&#39;s Your Fault</a>&quot;. In fact, there was even a <a href="https://www.reddit.com/r/programming/comments/qbg9y/i_broke_your_code_and_its_your_fault/">lengthy reddit discussion</a> about it in which folks tried to decide if I was clinically insane, or just a regular moron.</p> <p>Hyperbolic title aside, I still stand by the original point. Even if someone else does something as annoying as change the interface I was depending on in my code, it shouldn&#39;t be possible for them to so thoroughly break the code I wrote without SOMETHING telling them that they did so. <strong>All it takes is one failed test to say &quot;hold up, you broke shit.&quot;</strong></p> <p>When I push code up to the shared repository, it&#39;s my job to ensure it works, not QAs. But it&#39;s also my job to ensure that a junior engineer or a new hire can&#39;t just break it without something telling him or her it happened. When I write code, I try to imagine, what would happen if some other engineer came in and modified the class I just wrote, maybe didn&#39;t understand why I was doing <code>-1</code> somewhere, and so they just removed it? Would that be an annoying thing to do? Sure, and I would hope that the other engineer might ask me why I was doing it if I failed to make it obvious from the code itself. But maybe this is years from now and I&#39;m not even at the company anymore, so they remove the <code>-1</code>, or they think my code sucks so they rewrote the entire function from scratch. The instant they do that, a test I wrote somewhere should fail (hopefully with an explanation of why it needed to be the way it was).</p> <p>By writing my code like this, and creating what reddit argued is <em>too many tests</em>, I am encouraging the other members of my team to embrace <strong>fearless refactoring</strong>. Don&#39;t like how I wrote something? Refactor it, and don&#39;t worry about breaking anything - I wrote enough tests to ensure that you can&#39;t. Is it possible I&#39;ll make a mistake and fail to cover something I should have? Of course it is, but when this happens, the refactor-er in question did me a favor by highlighting a mutation that I missed.</p> <p>Top comment on that thread questions the wisdom of being happy that an app breakage highlights a missing test we can add. The commenter says to try telling the client about your unit test suite while they&#39;re losing customers left and right due to the bug. I guess that&#39;s a fair poi-- wait, what? You&#39;re developing applications where breakages and bugs can utterly destroy your company, and you&#39;re <em>not</em> writing a metric ton of tests? That&#39;s some serious Evel Knievel shit right there. Um, Evel Knievel was a stuntman in the 70&#39;s. Er, the 70&#39;s were a decade about 30 years before Spongebob first aired. Nevermind.</p> <table class='image alignright' width='306'><tr><td><img src="http://www.nomachetejuggling.com/assets/safetynet.jpg" width='300' height='300'/></td></tr></table> <p>Look, the safety net of an overabundance of unit tests combined with some high-level smoke tests to ensure that basic functionality is always working should give the entire team the freedom to refactor and rewrite anything they don&#39;t like. If everyone on a team is able to adopt this attitude, the end result is code that is incredibly clean. <strong>If the team isn&#39;t fearlessly refactoring, and they&#39;re afraid to make tiny changes and improvements because something might break somewhere, your team is hamstrung</strong>. Modules start to have a &quot;here be dragons&quot; vibe, with everyone afraid to improve them and so they rot until your entire codebase is rotten and you think you need to rewrite it (we talked about that <a href="http://www.nomachetejuggling.com/2016/06/15/guidingprinciples-part1/#toc-the-team-unqualified-to-refactor-is-unqualified-to-rewrite">already</a> though).</p> <p>I&#39;m not saying it should be impossible to break my code. Changing the interfaces of things I depend on, or literally going in and modifying what I wrote could easily make it behave incorrectly. I can&#39;t stop that. I&#39;m saying it <strong>should be impossible to break it without a test automatically telling you that it happened</strong>.</p> <p>When you actually imagine that another engineers might come in and accidentally (or maliciously) modify your code, your tests get much stronger. You&#39;ll find that your assertions are better when you try to guard against this sort of thing, which is really what unit testing is all about. Lots of people track coverage for tests, but coverage basically just counts lines hit during the testing phase. You could write a suite of unit tests that actually hits every single line of code, giving you 100% code coverage, but makes no assertions whatsoever. Your coverage is high, but your tests are borderline useless in this case. <strong>Raw coverage isn&#39;t what I&#39;m talking about here</strong>.</p> <p>It&#39;s not about how many tests you have or how many lines they cover, it&#39;s about how strong the tests you have are. And approaching your tests with the attitude that it should be impossible to break your code without a test failing is how you make them strong.</p> <h2>Zealotry</h2> <p>This is probably the <em>strong opinion</em> I hold that comes closest to zealotry for me. As a counterexample, I really <a href="http://www.nomachetejuggling.com/2009/02/21/i-love-pair-programming/">love pair programming</a> but I left the gig where I did it regularly and took a job where the team really didn&#39;t like pairing, and I adjusted fine to not pairing. I usually write my tests first and enjoy the TDD red-green-refactor cycle, but there are times when I suspend this practice and write tests later. There are plenty of things I really love doing that I&#39;m more than happy to stop doing as the situation demands, but I don&#39;t think I can go back to not testing at all, and I might be unwilling to listen to arguments to convince me to.</p> <p>At this point in my career, the level of physical discomfort I feel writing code with no tests at all is unbearable. Not too long ago I was extremely busy with one task but was forced to switch gears to implement a small change I didn&#39;t really agree with to an unrelated part of the code. As some kind of juvenile form of protest, I half-assed the code and wrote no tests, just to get it done and off my plate so I could go back to what I was doing. I pushed it up to the central git repo and felt so uncomfortable with what I had done that I lasted about 60 seconds before going back in and writing some tests to cover the change and explain why in the test case. My rebellion was brief, I am not a badass.</p> <p>I&#39;ve heard of places where bosses will declare that unit tests are a waste of time that slow down development, and I genuinely don&#39;t think I could work in a place like that anymore. Ten years ago and I wouldn&#39;t have cared, but today it just seems like an impossible request that I don&#39;t write tests, like asking me to drink lighter fluid or something. I&#39;ve fallen into such a comfortable cycle of code-a-little, test-a-little that eschewing the process feels completely unnatural and foreign; whiteboard coding interviews seem so bizarre to me now, I&#39;d never write so many lines of code without tests at work. My god, there&#39;s an <code>if</code> statement in it, that&#39;s two tests!</p> <p>My code design has vastly improved by thinking about testability. Once upon a time I&#39;d have used <code>Math.random()</code> or <code>System.currentTimeMillis()</code> or <code>new FileReader(&quot;whatever.txt&quot;)</code> without a second thought, but viewing code through the lens of testability made me realize that all of those things are subtle integration dependencies on the underlying system. Figuring out how to write unit tests for code that depends on random number generators, a clock, or the filesystem has forced me to consider things as candidates for dependency injection that I&#39;d never have considered without those tests. Even if I were to delete those tests afterwards, the code is still cleaner and better for having been designed with them in mind.</p> <h1>If You Hate It, Do It More</h1> <p>This one is easy to say, but very hard in practice to commit to. Basically, whenever I find myself dragging my feet on something I don&#39;t want to do, I need to sit down and ask myself why I hate doing it. Chances are, when I get to the root cause of my disdain or anxiety, I find that it&#39;s because something is extremely inefficient or error-prone.</p> <p>Hate performing deployments? Why? There&#39;s a good chance it&#39;s because it involves a bunch of manual steps, handbuilding artifacts and manually uploading them somewhere, then shelling into multiple boxes and executing commands. The desire to get away from anything unpleasant is very strong, but it&#39;s these situations that would benefit the most from doubling down and doing it more often.</p> <p>If you&#39;re deploying every quarter because it&#39;s such a pain, you need to start deploying every month. If you still hate it, every week. If you still hate it, every day. At some point you&#39;ll hit a point where you say enough is enough, and if you&#39;re going to deploy this crap every day then it needs to be easier. And that&#39;s when you start developing deployment pipelines and writing automated scripts. <strong>The more you do something you hate, the better you&#39;ll get at doing it</strong>, if only to keep your sanity.</p> <table class='image alignleft' width='306'><tr><td><img src="http://www.nomachetejuggling.com/assets/hate.jpg" width='300' height='196'/></td></tr></table> <p>Hate provisioning machines? Start adding and removing boxes from clusters on a regular basis. At first it will be difficult and annoying - <em>that&#39;s good</em>, that&#39;s what will make you better. In no time you&#39;ll be using OpenStack or AWS, augmenting setup with Puppet or Chef, or maybe even containerizing your entire process with Docker. Your infrastructure will be better for it, everything that you hate doing is likely a weak spot in your development.</p> <p><strong>Hating something is your brain&#39;s way of telling you &quot;this sucks,&quot; but instead of responding by hating it, respond by taming it.</strong> The more you do it, the easier it is to figure out which parts suck the most, and how you can improve them.</p> <p>One of my favorite examples of this is Netflix&#39;s <a href="http://techblog.netflix.com/2012/07/chaos-monkey-released-into-wild.html">Chaos Monkey</a> approach. Dealing with failure was such a negative experience for Netflix that they started doing it all the time, so often that they built software that would randomly fail-out nodes, clusters, or even entire regions. It forced Netflix to revisit how their software works, and handle failure better. What came out the other end was a vastly superior product. And also &quot;Daredevil&quot;.</p> <p>This principle is tough because it&#39;s a lot like cleaning an incredibly messy room or a trainwreck of a garage. Things start pretty bad, but the real issue is that <strong>things have to get worse before they can get better</strong>. Only by embracing the things you hate doing the most do you force your own hand, resulting in something that can do the horrible job you hate automatically, on-demand, and quickly.</p> <h2>Meetings</h2> <p>Yes, this principle applies to basically every aspect of your job. This one is particularly tough for me but: if you hate meetings, have more of them. Start having daily meetings if you need to. In so doing, you and the rest of the team will discover exactly what it is you hate about meetings so much. The only way to really figure out EXACTLY what you hate about meetings is to expose yourself to them so often that it becomes immediately apparent what doesn&#39;t work about them for you.</p> <p>Once you&#39;ve identified what meeting dysfunctions make you despise them so much, it&#39;s easier to fix those things and make meetings more enjoyable. Honestly, I hate meetings too but I need to ask myself: geeze, why? Should it really be so unpleasant to meet and chat with other engineers I respect and enjoy working with? Are we really such misanthropic jerks that we can&#39;t enjoy exchanging ideas? And don&#39;t say that the reason you hate meetings is because they prevent you from doing <a href="http://www.nomachetejuggling.com/2012/10/05/getting-real-work-done/">Real Work</a>, I&#39;ve already talked about how dumb that is.</p> <p>After you and your team realize what doesn&#39;t work about meetings, you can take steps to address them until meetings aren&#39;t something you despise. And once you don&#39;t hate it, the inverse of the rule applies: <strong>if you like it, you can survive doing it less</strong>. Dial your meeting schedule back down once the thing you hate is <em>not meeting</em>.</p> <h1>Be the Worst Person in the Band</h1> <p>I got this from Chad Fowler&#39;s &quot;<a href="https://amazon.com/Passionate-Programmer-Remarkable-Development-Pragmatic-ebook/dp/B00AYQNR5U/">The Passionate Programmer</a>&quot; who in turn took it from jazz guitarist Pat Metheny, who said:</p> <blockquote> <p>Always be the worst guy in every band you’re in.</p> </blockquote> <p>This idea has resonated with me ever since. Is it uncomfortable to be the worst person on the team? Yeah, it sure is. And it&#39;s this discomfort that will drive you to be better. When you&#39;re the best person in the band, you walk around with tons of confidence but you aren&#39;t learning anything and you aren&#39;t improving, because nothing is driving you to. When you&#39;re the worst, you have to step it up.</p> <p>One of the great things about this career is that it&#39;s absolutely impossible to ever know all of it. It&#39;s growing and new tools and ideas are being added at a rate faster than you can possibly learn them. I can see this being stressful for some people, but it&#39;s my favorite thing about it. There&#39;s always, <strong>always</strong> more stuff to learn and improve. It&#39;s like being a bookworm and walking into a library of infinite size.</p> <table class='image alignright' width='306'><tr><td><img src="http://www.nomachetejuggling.com/assets/yoko.png" width='300' height='205'/></td></tr></table> <p>Nothing makes me want to learn more and be better than being surrounded by people who are better than me. I&#39;ve worked plenty of jobs where I was the worst guy in the band, and plenty where I was the best, and I always come out of the ones where I was the worst guy in the band feeling like I just spent the entire time leveling up like crazy. Being the best fills you with confidence, which is nice on an emotional level, but it&#39;s nowhere near as satisfying as coming out the other end of a job a vastly improved person. </p> <p>I&#39;ve modified this slightly to <strong>be the second worst guy in the band</strong>. Being the truly worst can make you feel useless, like you&#39;re not making any valuable contribution. Plus it actually helps to be able to mentor someone, one of the most effective ways to learn something is by teaching. In any case, definitely don&#39;t be the best person in the band.</p> <p>Another way I&#39;ve heard this phrased comes from Scott Bain as quoted in <a href="https://smile.amazon.com/Beyond-Legacy-Code-Practices-Software/dp/1680500791">Beyond Legacy Code</a>:</p> <blockquote> <p>Always strive to be mentoring someone and to be mentored by someone.</p> </blockquote> <p>You can subscribe to all the blogs, read all the books, and attend all the conferences, but nothing will help you learn and keep up with the ever-changing world of software development like working every day with someone better than you. The more people that you&#39;re working with that are better than you, the stronger this effect. </p> <h1>Your First Loyalty is to Your Users</h1> <p>This one might be a little controversial, and proudly proclaiming it on my blog might make me unemployable. But at the end of the day, I as a software engineer answer to an authority greater than my product owner, my boss, my VP, my CTO, or anyone else who signs my paychecks: I owe my users quality software. If I wouldn&#39;t be willing to attach my personal cell phone number to the feature I&#39;m developing, I shouldn&#39;t write it.</p> <table class='image alignleft' width='306'><tr><td><img src="http://www.nomachetejuggling.com/assets/tron2.jpg" width='300' height='169'/><p class='caption'>I fight for the users</p></td></tr></table> <p>I have, on more than one occasion, gotten into a heated debate with a product owner or even a supervisor about a feature I was asked to implement. Often, this stems from situations where the people who are USING the product aren&#39;t the ones PAYING for it, and the client&#39;s higher-ups are writing the checks for features that their underlings using the product might dislike. I&#39;ve found myself usually able to win these arguments by helping the product owners understand how unhappy their users will be, and pointing out that happy users will eventually leave their current company and become a sales lead at their next gig, but the most heated and intense arguments I&#39;ve been involved in at work always stemmed from me advocating on behalf of the voiceless users who would end up on the receiving end of antagonistic features.</p> <p>Unlike most the other principles on this list, this one won&#39;t result in better quality codebases or more SOLID or testable designs - it actually affects the product I build at a business level. I will never do anything half-assed, never lie or mislead my users, never take advantage of them, and never intentionally create a negative experience for them because it will line my or someone else&#39;s wallet. This is especially true when my end-users are not engineers themselves - <strong>they have no power or control in this software-centric world, so taking advantage of the power imbalance is particularly unethical</strong>.</p> <p>I&#39;m not arguing that everything you build needs to make the world a better place at some cosmic level, or that you need to be carbon neutral in every facet of your life or anything like that. I understand that sometimes you need to pay the bills. But what I&#39;m saying is to never, ever forget that at the end of the day some poor schmuck is going to be using the thing you&#39;re building, and this person has people who care about him or her as much as you care about your loved ones. Imagine your mother or husband or best friend using your software - do you feel good about yourself? If not, don&#39;t build it.</p> <p>Don&#39;t write software that <a href="https://en.wikipedia.org/wiki/Volkswagen_emissions_scandal">tricks emissions tests</a> just because your asshole boss told you to. Don&#39;t write <a href="https://en.wikipedia.org/wiki/Sony_BMG_copy_protection_rootkit_scandal">copy protection schemes that phone home with a user&#39;s private data</a> just because your CEO thinks he&#39;s entitled. Don&#39;t develop code that <a href="http://www.cnet.com/news/e-tailers-snagged-in-marketing-scam-blame-customers/">opts users into monthly charges if they are dumb enough to trust you when using your product</a>, there&#39;s no such thing as a &quot;stupid tax&quot;, <strong>your stupidest users need your advocacy the most.</strong></p> <p>Just remember, someday there might be a scandal and a court case that involves engineers being held accountable for the features they built, and &quot;I was just following orders&quot; may not be enough to save you. Be proud of what you create. It&#39;s not enough to assume <a href="https://groups.google.com/forum/#!msg/comp.lang.c++/rYCO5yn4lXw/oITtSkZOtoUJ">the guy who ends up maintaining your code will be a violent psychopath who knows where you live.</a> - assume that your poor users are as well.</p> <table class='image alignright' width='306'><tr><td><img src="http://www.nomachetejuggling.com/assets/uncle-ben.jpg" width='300' height='226'/></td></tr></table> <p>I think there&#39;s a tendency for developers to &quot;just do what they&#39;re told.&quot; The users and their experience is the concern of the product owners, marketing types, salespeople, and other stakeholders - the developers just build the software according to the requirements, right?. In all honesty, I wish that this was a safe mindset to adopt - I&#39;d rather concern myself only with the code and my fellow developers who have to work with it, and leave the features and user experiences up to other people. But time and time again, I&#39;ve found that for whatever reason the folks in those positions lose sight of the user experience and request antagonistic features. At the end of the day, the engineer is where the rubber meets the road - we&#39;re the gatekeepers on what actually gets created, so <strong>we&#39;re the last line of defense before something goes out the door that will make the lives of users worse</strong>. Product Owners and marketers can draw boxes and do photoshop mockup designs all they want, but the engineers are the only ones with the <em>power</em> to actually build the stuff users will be interacting with, and as the sole wielders of this power, we have the <em>responsibility</em> to consider those users even when others don&#39;t.</p> <h1>Conclusion</h1> <p>I feel like there are more things I wind up saying a lot, but one of the most challenging parts of writing up this list was even stepping back enough to realize which things could be written down. When you live by certain ideals long enough, they become so ingrained that it&#39;s hard to even remember what the principles are. Most of the ones on this list I realized only because I&#39;ve been called out by other people for saying them so often.</p> <p>Anything missing? Any principles that you live by as an engineer? Leave a comment, I&#39;m curious what other people see as their <strong>Software Engineering Golden Rules</strong>.</p> Mon, 20 Jun 2016 00:00:00 -0400 http://www.nomachetejuggling.com/2016/06/20/guidingprinciples-part2/ http://www.nomachetejuggling.com/2016/06/20/guidingprinciples-part2/ Software Engineering Guiding Principles - Part 1 <p>I find that I repeat myself often at work. There are a handful of things I say so often when discussing decisions that I&#39;ve been called out for it on occasion for acting like a broken record.</p> <p>But the reason I keep repeating these phrases is that I think they inform a great deal of my decision-making. They are, in effect, my guiding principles when developing software professionally.</p> <p>I thought it might be fun to write a few of these things down because I think that they&#39;re worth sharing - I feel like these principles have steered me in the right direction time and time again. Obviously, there are exceptions to these and there are times when they should be ignored (after all, not being a zealot is one of the principles) but I think they will generally take an engineer down the right path.</p> <h1>Have Strong Opinions, Weakly Held</h1> <p>I think the phrase I&#39;ve heard more than any other in my life is &quot;tell us how you really feel!&quot; which is I guess people&#39;s way of telling me I&#39;ve made them uncomfortable by expressing an opinion too aggressively. It&#39;s true, I can be very strongly opinionated, and I&#39;ve gotten into more than my fair share of, oh, let&#39;s call them &quot;passionate discussions&quot; in the workplace. I&#39;m never insulting or personal, but I have strong opinions on how to do things.</p> <p>That said, I think it&#39;s important to always be open to having my mind changed. If anything, I think I&#39;m TOO easy to convince to change my mind on something, often it takes only one strong counterpoint to completely demolish an opinion I&#39;ve held firm to for years. My opinions are informed by years and years of experience, but that experience doesn&#39;t always apply in every situation, so it&#39;s important to be willing to adjust in light of new information or facts.</p> <p>Apparently this phrase &quot;strong opinions, weakly held&quot; comes from Stanford Professor Bob Sutton. I think it&#39;s a good way to approach every opinion really. I&#39;ve switched between polar opposite positions on a number of issues, including political and philosophical issues that I won&#39;t get into on this blog, but I think I do a good job of allowing my convictions of experience to be suspended to make way for alternative arguments. <strong>I never assume I&#39;m objectively right just because I care</strong>.</p> <table class='image alignright' width='306'><tr><td><img src="http://www.nomachetejuggling.com/assets/zealot.jpg" width='300' height='259'/><p class='caption'>Unless you hunger for battle, don't be a zealot</p></td></tr></table> <p>It&#39;s important that the thing that makes an opinion weakly held is a strong, rational, logical argument for the alternative position. I won&#39;t back down on something I think is important because of how passionately another person disagrees, or how upset it makes them that they&#39;ve met opposition. This is what makes the opinion strong: I genuinely care about believing the largest possible number of true and correct things, so the only way to dislodge a strong opinion is with true and correct things that work to counter it.</p> <h2>Don&#39;t Be A Jerk</h2> <p>I cringed when I watched Season 3, Episode 6 of my favorite show Silicon Valley, as the main character Richard felt so strongly about Tabs over Spaces that he alienated everyone in his life over it. These debates are so incredibly pointless to me, I do not understand how people waste so much time caring about them. <strong>Strong opinions are not the same as zealotry</strong>, zealotry is company and team poison. Strong opinions only matter if the things they&#39;re about matter. Having extremely strong opinions about tabs vs spaces, or emacs vs vim makes you borderline un-hireable to me, bringing zealots onto your team violates the <a href="https://smile.amazon.com/Asshole-Rule-Civilized-Workplace-Surviving/dp/0446698202">No Asshole Rule</a> (though for the record, spaces and vim, #sorrynotsorry).</p> <p>Additionally, it&#39;s fine to have strong opinions but if you find yourself belittling or mocking other people in order to stand by them, they probably aren&#39;t that strong. Your positions on technical matters should stand on their own weight, without needing to knock people down. Don&#39;t be one of those people that walks around acting like a jerk and then justifying it by saying you have strong opinions. The best engineers I&#39;ve worked with have consistently been skilled at <strong>not only having well-reasoned strong opinions, but communicating those opinions respectfully to others.</strong></p> <p><span data-pullquote="It's better to have a hole in your team than an asshole. " class="left"></span></p> <p>Being a technical wizard doesn&#39;t give someone the right to be a pompous ass to everyone else. I&#39;m a strong advocate of taking people who are, at a personal level, insufferable, and firing them for being a poor cultural fit, regardless of how much they know about this or that technology. It&#39;s better to have a hole in your team than an asshole.</p> <p>I started this list with this one in particular because it&#39;s important. The rest of this list is, essentially, a list of strongly held opinions I maintain. But it&#39;s important that even these opinions, having reached Guiding Principle level, are subject to change in the light of strong counterarguments, or subject to suspension in light of unique circumstances.</p> <h1>The Team Unqualified to Refactor is Unqualified to Rewrite</h1> <p>I strongly, strongly believe that a full-on code rewrite is nearly always the wrong thing to do. Either you pull everyone off the current iteration of the product to do the rewrite, which means your main product languishes, or you pull some people off to do the rewrite, meaning the rewrite team has to always be catching up with the ever-growing main product.</p> <p>From a simple project management standpoint, this is a disaster. Want to know how long the rewrite will take? Well, in the former case, you&#39;re working with a team that&#39;s dealing with new technology and new development, so there&#39;s no way to apply any previously recorded team velocity as a prediction of future performance. Moreover, you don&#39;t actually have any sense of the scope of the project, because the requirements are basically &quot;everything the app does now&quot;, which will include weird corner cases that have long since been forgotten. So you have an unknown scope and an unknown team velocity, and you&#39;re trying to make a prediction of when this work will be completed? So development is going to stop on the main product line for an indeterminate amount of time. And this is the BEST case scenario, the one where everyone can focus on doing the rewrite.</p> <p>In the latter case, it&#39;s even more unpredictable - you still have the unknown scope issue, but it&#39;s worse because you also have to include, in the scope, getting to parity with whatever else is built while the rewrite is being worked on. If the rewrite would take 3 months, you have 3 months worth of new features on the main product to catch up to. If it would take 6 months, you have 6 months of features to catch up on. And since you don&#39;t know how long it will take just to reach current parity, you can&#39;t predict how far in the hole you&#39;re going to be when it&#39;s &quot;done&quot;, which means it adds ANOTHER layer of unknown time into the mix. Maybe adding those 6 months of features takes you 5 months, so when you&#39;re done you&#39;ve got another 5 months to catch up on. That 5 months of work takes you 3 months to complete, so you have another 3. You&#39;re basically asymptotically approaching done. And remember, the velocity of the &quot;main product&quot; team will be affected by the loss of resources who peel off to do the rewrite, so you have little sense of the velocity of not one, but both teams. If you know your car&#39;s speed, you can predict when it will pass a landmark - but you can&#39;t possibly know when it will pass another moving car if you don&#39;t also know that car&#39;s speed perfectly. If you know neither car&#39;s speed, you&#39;re utterly done for.</p> <table class='image alignleft' width='306'><tr><td><a href='https://www.flickr.com/photos/8047705@N02/5463789169/in/photolist-9jPmax-9DDYgg-nuZtxE-qhFho-ejtid-6c7xeY-ze5cZ-nmKFqr-b1AcSK-9FVdCx-b1AhTZ-b1A75H-dZkGN1-nB5JZn-qhFau-2DKnb-f4N574-cBqwpb-dD7wZD-5cTVaL-zCEF8-9F679e-ogasoM-aDd8E1-9bBdFG-4LeSwt-aDd8s9-J1y1LF-aD9gKD-Curwec-8MGTB8-9cP1vZ-dfCV95-HHBNMQ-oqYWpW-CUmf1-6Skbcz-5SptQa-5qD4Md-4mtSKC-eftGLt-8P76u6-oHB4w9-F8KKsW-7pZA7u-9XRFMM-tPAFC-7xYyVJ-6YmJXf-9BVWnR'><img src="http://www.nomachetejuggling.com/assets/sorry.jpg" width='300' height='199'/></a><p class='caption'>Go back to start</p></td></tr></table> <p>Moreover, from an engineering standpoint this is a terrible idea. Everyone likes doing greenfield work because it&#39;s new and exciting, but you have to ask, why do the engineers want to avoid maintaining and refactoring the existing product? Is the codebase such a spaghetti mess that it&#39;s too difficult to add anything, so the team wants to try again from scratch? <strong>Who the hell do you think made that dumpster fire in the first place?</strong> Why on earth would that same team suddenly do it right the second time around? Especially when under the pressure of &quot;we have to get caught up&quot; and the time-pressure of the company&#39;s primary software products being frozen or at least slowed while the team develops it? It&#39;s even MORE likely that corners will be cut and quality will suffer, not less likely.</p> <p>Refactoring the codebase is almost always the right way to go. Take the awful parts that you want to rewrite and slowly but surely refactor them into the clean codebase you want. It might take overall longer to be &quot;done&quot; with the effort, but the entire time it&#39;s happening the main product is still in active development without the &quot;two cars racing&quot; situation. Refactoring code is, though slower, also easier to do than rewriting it from scratch, because you&#39;re able to do it in small steps with (hopefully) the support of a huge test suite to ensure you don&#39;t break anything. <strong>Since refactoring is easier than rewriting, any team that says &quot;it&#39;s too hard&quot; to the idea of refactoring the existing codebase instead of rewriting it is inherently not good enough to do the rewrite.</strong> The end result will actually be worse.</p> <h2>Exceptions</h2> <p>There are a couple noteworthy exceptions to this. One, when the reason for the rewrite is a complete change in technology, specifically the language of implementation. If you&#39;re working with Java and want to rewrite in Scala or Clojure, the team should be able to refactor piece by piece since it all compiles to the same bytecode. However, if the team needs to move from a dead technology such as ColdFusion to something else like .NET, a full rewrite is the only way to go. This may also apply in the case of using a prototyping technology to develop the first iteration of a product, only to discover that there&#39;s no way to make the system scale, such as in the case of Twitter&#39;s abandonment of <a href="http://www.gmarwaha.com/blog/2011/04/11/twitter-moves-from-rails-to-java/">Rails in favor of Scala</a>. Not every company has the resources to <a href="http://readwrite.com/2010/02/02/update_facebook_rewrites_php_runtime_with_project/">develop a new PHP runtime</a> just to avoid rewriting their codebase in something other than PHP, sometimes you have to bite the bullet and pick different technology.</p> <p>Another exception is when you find yourself in an &quot;over the wall&quot; situation. Perhaps a team of contractors or consultants or offshore engineers were hired to develop the first iteration of a project, and then the codebase was tossed over the wall to another team to maintain. In this case, the new team may in fact be qualified to both refactor OR rewrite the codebase, and may simply decide the codebase as-is is too much of a mess to bother with and do a rewrite. In this instance, I still would encourage exploring every possibly opportunity to refactor first, but believe me when I say I&#39;ve been on the recieving end of these codebase bombs enough to fully appreciate that sometimes you just need to rewrite the whole thing.</p> <p>One more exception, if your &quot;product&quot; is mostly just a collection of microservices and you&#39;re talking about rewriting some of them, that&#39;s another story. In the land of microservices, rewriting a service essentially <em>is</em> refactoring, and presumably you have a collection of integration-style tests against each microservice, so a rewrite can be done relatively quickly and relatively safely. Even if you want to rewrite all of the services, you&#39;re able to do it one at a time - this is one of the big advantages of microservice architectures.</p> <h1>Choose Boring Technology</h1> <p>I really can&#39;t say this any better than Dan McKinley&#39;s original post <a href="http://mcfunley.com/choose-boring-technology">Choose Boring Technology</a>. In it, McKinley argues that every team or company should start out with three innovation tokens. You can spend these tokens whenever and however you please, but they don&#39;t replenish quickly. Every time you pick an exciting or buzzwordy or cutting edge technology instead of an old standard, you spend a token.</p> <p>Relational Databases are boring. Java is boring. JQuery is boring. Apache is boring. Linux is boring. Tomcat is boring. Choose something &quot;cool&quot; instead of something boring, and you&#39;ve spent an innovation token. Boring technology is boring because it&#39;s <em>known</em>, not because it&#39;s <em>bad</em>. Its failure modes are understood, and it probably has a host of libraries and support tools make it easier to live with in the long term.</p> <p>There&#39;s nothing wrong with Java, tons of scalable applications have been built on Java, and &quot;it&#39;s boring&quot; isn&#39;t a good enough reason to choose something else. If your team truly feels like Scala or Clojure or Erlang or whatever is the right tool for the job, by all means use it, but that&#39;s one innovation token spent. Pick MongoDB over MySQL or Oracle and you&#39;ve got one left. Any time you COULD use technology you&#39;re already using (&quot;our other codebase is .NET&quot;) but decide to pick something new instead, you spend a token.</p> <p>Boring Technology is easy to pick up, easy to research, easy to debug, and frankly easy to staff for. I&#39;m sure the engineering team is happy to pad their resumes with cool buzzwords while simultaneously making themselves irreplaceable, but is that really the best thing for the product and the company? When boring technology fails you, there are stacks of books and internet forums available to assist you - there&#39;s nothing worse than the feeling of excitement you get when you search for your error message and find that someone else has had the EXACT same problem as you before, only to be followed by the crushing blow of zero replies.</p> <script async class="speakerdeck-embed" data-slide="47" data-id="454e3843ac184d3f8bcb0e4a50d3811a" data-ratio="1.31113956466069" src="//speakerdeck.com/assets/embed.js"></script> <hr> <p>I&#39;ve worked plenty of jobs where the team was building plain old Java Web Applications using Spring, backed by MySQL or Oracle databases. You know what? Those products worked just fine. Did the teams have the <em>most</em> fun in the world writing that code? No, probably not, but we got the job done and the products performed quite well (and were easy to fix when they didn&#39;t). A buddy of mine is fond of watching engineers pick and choose cool technologies out of the pool of the latest-and-greatest, only to remind us that he worked on a 911 call routing application written in Java with a MySQL database, and it ran just fine saving tons of lives.</p> <p><span data-pullquote="It's not about how much fun I have. " class="right"></span></p> <p>At my current gig, we decided to build a 150,000-line codebase using Scala. Scala seemed like the right tool for the job, given the particular constraints we had about scalability and throughput in the system. I like Scala a lot, and there&#39;s no doubt that we&#39;ve made tremendous productivity gains by utilizing features exclusive to Scala, but if I&#39;m truly honest with myself did we actually make an overall <em>net</em> productivity gain? When you factor in time lost trying to understand confusing code, time lost by the compiler doing a <a href="https://wiki.scala-lang.org/display/SIW/Overview+of+Compiler+Phases">twenty-pass compilation</a> (holy shit), and time lost by having to manually perform refactorings that our IDEs couldn&#39;t automate due to weak tooling support, I&#39;m not actually sure we came out ahead. Especially given Java 8&#39;s functional programming features, I&#39;m not sure I&#39;d bother picking Scala over Java 8 today, as much fun as I have working with it. It&#39;s not about how much fun I have.</p> <p>Ultimately, it&#39;s really not about me or how much I enjoy working with particular tools and technologies. My job isn&#39;t to have a blast, hell it&#39;s not even really to &quot;write code&quot; - my job is to solve business problems, and it so happens the best tool I&#39;m most competent using for that is code. It&#39;s important to stay up to speed on the latest and greatest technologies so that you as an engineer have the knowledge to know when it&#39;s time to spend an innovation token, but honestly I think most of that effort should be relegated to conference attendance, reading, and personal github accounts. Don&#39;t make company decisions based on how many buzzwords you can add to your resume.</p> <h2>Inventing Languages</h2> <p>I&#39;d like to add that &quot;writing your own programming language&quot; should be worth four innovation tokens all on its own. If you develop an in-house programming language, you&#39;d better have a staggeringly good reason. Good programming languages are hard to write, and unless you have a number of Computer Science PhDs with specializations in Programming Language Design and Implementation on the team, chances are all you&#39;re actually doing is writing an overly complex DSL. The kind of thing whose compiler/transpiler/transliterator fails with &quot;syntax error somewhere&quot; in the event of a mistyped character, rather than a helpful diagnostic and a line number.</p> <p>Don&#39;t create your own programming language. Your language will be weak, your tools will be poor, and language support within other tools will be nonexistent. You probably aren&#39;t going to properly staff the design and support of the language you&#39;ve created. Unless you have an entire team of people devoted exclusively to maintaining that language and writing Eclipse plugins for it or whatnot, your technical debt is so crater-like that you can&#39;t even tell you&#39;re standing in a hole because it extends past the horizon. Whatever huge productivity gains you think your new language is offering your team, they&#39;ll be canceled out and then some.</p> <p><strong>99 times out of 100, a new language isn&#39;t what you want to build, but a library or a framework is</strong>. By all means, develop those in house if need be (but staff their development). Unless you&#39;re developing a language as part of your core business, like Apple developing Swift, don&#39;t do it.</p> <h1>Will You Understand This at 3AM?</h1> <p>Frequently John Carmack is cited as an example of an eccentric genius, the kind of guy who is way ahead of his time. I have to admit, I&#39;m also in awe of a great deal of what he&#39;s done with code. Take this square root function he wrote for Quake III arena:</p> <div class="highlight"><pre><code class="language-c" data-lang="c">float Q_rsqrt( float number ) { long i; float x2, y; const float threehalfs = 1.5F; x2 = number * 0.5F; y = number; i = * ( long * ) &amp;y; // evil floating point bit level hacking i = 0x5f3759df - ( i &gt;&gt; 1 ); // what the fuck? y = * ( float * ) &amp;i; y = y * ( threehalfs - ( x2 * y * y ) ); // 1st iteration // y = y * ( threehalfs - ( x2 * y * y ) ); // 2nd iteration, this can be removed return y; } </code></pre></div> <p>But notice line 10, <code>i = 0x5f3759df - ( i &gt;&gt; 1 );</code>? It&#39;s easy to find, because it&#39;s elucidated with the helpful <code>what the fuck?</code> comment. There&#39;s no doubt that this code is extremely clever, and it&#39;s beyond question that it&#39;s extremely fast. It also requires an entire <a href="https://en.wikipedia.org/wiki/Fast_inverse_square_root">2000-word Wikipedia article</a> to understand.</p> <p>In fact, Carmack himself wasn&#39;t even the creator of this bit of wizardry, it came from Terje Mathisen, an assembly programmer who had contributed it to id Software previously. And in fact, he likely got it from another developer, who had gotten it from someone else. This is why the comment <code>what the fuck?</code> is right there - nobody understood it. And yet there it was, pasted into the Quake III engine code because it seemed to work and it was fast. Obviously this worked out for id, and <a href="https://www.youtube.com/watch?v=PcbpIntnG8c">Quake III is awesome</a>, but it probably wasn&#39;t the wisest idea to stake their company&#39;s product on code that nobody understood.</p> <p>Was it clever? Absolutely. <strong>But <a href="https://simpleprogrammer.com/2015/03/16/11-rules-all-programmers-should-live-by/">clever is the enemy of clear</a>.</strong></p> <p>I try not to ever write comments in my code. Comments should not be used to explain how something works, that should be apparent from the code itself. And if that means adding a few temporary variables so that their names can be helpful (or inspected while debugging), or having some comically long method names, so be it. Often people say that comments can be used to explain &quot;why&quot; something works instead, but frankly I find that a few unit tests for the code in question will do a better job of explaining the why than a comment ever could - at the very least, take the comment you&#39;d write explaining why and make it the name of the test. <strong>Code is for <em>what</em>, tests are for <em>why</em>. Comments are for jokes.</strong></p> <p>Obviously it&#39;s difficult not to be proud of yourself when you&#39;ve gotten some long method down to a one-liner (even if it is one incredibly long line) or invented some massively clever solution to a problem. And indeed, sometimes these clever tricks really are necessary to get the required performance out of a system (as in the Quake III square root example). That&#39;s why I&#39;ve found this heuristic so handy (hattip to <a href="http://neidetcher.com/">Demian Neidetcher</a>):</p> <p><strong>If your cell phone rings at 3AM because this code causes a production outage a year from now, will you be able to understand and reason about the code enough well enough to fix it?</strong></p> <p>Imagine that your job is basically on the line here, you&#39;re now in a conference call with your boss, your boss&#39;s boss, your boss&#39;s boss&#39;s boss, and the CTO. Hell, maybe the CEO is on talking about the millions of dollars in lost revenue every minute the product is offline. Your heart is racing from being startled awake, and your eyes are barely able to focus enough to read your laptop screen. Do you <em>really</em> want this to be what comes into focus in the middle of the night?</p> <div class="highlight"><pre><code class="language-scala" data-lang="scala">(n: Int) =&gt; (2 to n) |&gt; ( r =&gt; r.foldLeft(r.toSet)((ps, x) =&gt; if (ps(x)) ps -- (x * x to n by x) else ps) ) </code></pre></div> <p>Yes it&#39;s clever, yes it&#39;s fast, congratulations on how smart you are. But your company code repository isn&#39;t the place to show off your l33t coding ski11z, do that shit in your personal github account. You&#39;re not being paid to fluff your e-peen, you&#39;re being paid to solve the company&#39;s business problems, and that means writing something that can be understood by the other people they hired. Code&#39;s primary purpose is to be read by other human beings (<a href="https://mitpress.mit.edu/sicp/front/node3.html">and only incidentally for machines to execute</a>), otherwise we&#39;d all be writing directly in machine language. So if this future version of yourself won&#39;t understand the code just from being tired, what chance does the dumbest person on your team have of understanding it? Stop showing off, your job (and maybe even your employer&#39;s future) may someday depend on it.</p> <h1>Deliver Working Software Early and Often</h1> <p>I realize this is just a rewording of a standard part of the <a href="http://www.agilemanifesto.org/">Agile Manifesto</a>, and I could just as easily say &quot;Be Agile!&quot; here. But I think the truth is Agile has come to mean a lot of different things to a lot of different people, and has become a term so overloaded and hijacked that it&#39;s effectively become <a href="https://pragdave.me/blog/2014/03/04/time-to-kill-agile/">useless as a phrase</a>.</p> <p>I like most of the ideas of the Agile Manifesto, but I think the most important thing to take away from it is the unparalleled value of getting working software into the hands of users as quickly and frequently as possible. I absolutely detest when features are held back so that they can be released in a &quot;big bang&quot; to really wow and excite users (hey Product Owners, your users really don&#39;t care as much as you think, you&#39;re just building a thing they&#39;re forced to use to accomplish something). As long as a feature actually works end to end, get it into the hands of users and solicit feedback right away; every day you keep working code behind a gate is a day you give your competitors to steal users away from you. It&#39;s also a day that you are effectively lying to your users - the most important people to your software - about what your product is capable of doing. </p> <p>I despise long-running feature branches in version control as well, almost any time you want to make a branch I think it&#39;s better to make a feature flag that people (specifically, product owners) can turn on and off at will. Long-running branches are incredibly susceptible to <a href="https://en.wikipedia.org/wiki/Ninety-ninety_rule">the 90/90 rule</a>. And if two subteams wind up creating simultaneous long-running branches off the same mainline trunk, pack it in, you&#39;re done for. </p> <p>Every &quot;big bang&quot; release I&#39;ve been a (reluctant) part of has ended in some form of failure. People think that the software is mostly done and then the effort spins its wheels at the end, trying to &quot;harden&quot; the release and remove bugs. Or the software is finally delivered only to discover that <a href="https://en.wikipedia.org/wiki/Pareto_principle">80% of the users are only using 20% of the features</a>, meaning that a more targeted, earlier release of those top 20% features would have been a far better use of engineering time and resources. The other 80% is now just cruft in the codebase, making it more difficult to add features later on, and nobody is using it.</p> <h2>Plans are The Opposite of Working Software</h2> <p>I think a corollary to this rule is, don&#39;t sell your users on non-working software. I really hate the tendency for &quot;marketing&quot; to <em>need</em> delivery dates on software features so that they can start selling the features now, a situation I&#39;ve seen at company after company. Don&#39;t try to sell users on features you plan on delivering, even if you&#39;re nearly certain about when those features will be done (but, hint, you&#39;re probably less certain than you think). That&#39;s selling vaporware, anything can change between now and then causing those features to be shelved or to not work properly. Instead, deliver working software early and often, and let the marketing folks sell users on what features are actually <em>done</em>, because more stuff will actually <em>be</em> done due to the team not wasting tons of time coming up with estimates (<a href="https://www.happybearsoftware.com/all-estimates-are-still-lies">read: lies</a>).</p> <p><center> <blockquote class="twitter-tweet" data-lang="en"><p lang="en" dir="ltr">Just start referring to “estimates” as lies.<br><br>“how long will that take?”<br>“well, if I had to lie, a week?”</p>&mdash; Trek Glowacki (@trek) <a href="https://twitter.com/trek/status/636286667087851520">August 25, 2015</a></blockquote> <script async src="//platform.twitter.com/widgets.js" charset="utf-8"></script> </center></p> <p>Obviously sometimes there are occasions where people need some sense of how long something will take, most notably when the company is deciding between two different features to implement and they&#39;re performing an analysis based on their cost (though in my experience, rarely does this happen and usually both features are requested anyway). But for the most part, using some roadmap or a plan to inform the company on how to sell their products is a mistake - give engineers the time to properly implement features well, and then when the features are done sell people on them. And remember, <a href="http://www.bloomberg.com/news/articles/2016-05-18/this-5-billion-software-company-has-no-sales-staff">good software sells itself</a>.</p> <h1>Part 2...</h1> <p>I split this list into two posts for really no good reason aside from length. If you want more, check out <a href="http://www.nomachetejuggling.com/2016/06/20/guidingprinciples-part2/">Part 2</a>.</p> Wed, 15 Jun 2016 00:00:00 -0400 http://www.nomachetejuggling.com/2016/06/15/guidingprinciples-part1/ http://www.nomachetejuggling.com/2016/06/15/guidingprinciples-part1/ My StrengthsFinder Results <p>At work, the BigWigs paid for a bunch of employees, including myself, to take the <a href="http://strengths.gallup.com/110440/About-StrengthsFinder-20.aspx">Gallup StrengthsFinder</a> test. This test gives the taker a series of choices between two things that aren&#39;t exactly opposites, and you have to select which one you identify closer with. In the end, the test tells you which of 34 possible strengths are your top 5.</p> <p>I enjoy taking personality tests for fun, but the way that the aforementioned BigWigs were attaching tremendous levels of importance to the results of the test made me a bit weary. Personality tests can often have a horoscope vibe, where they all say something so nice about the taker that everyone who reads it says &quot;yep, that&#39;s me!&quot;.</p> <p>So before the test, I took a look at the 34 possible strengths that the test would identify. I figured they&#39;d all be things I liked, so that when the top 5 were output, the taker would like the results. To my surprise, there were a number of strengths, about 10 of the 34, that I would have been downright irritated if they appeared in my top 5. To an extent that, if the exercise told me any of those 10 were strengths of mine, I&#39;d be able to instantly disqualify the test as bunk.</p> <p>So I took the test with a skeptical eye toward it, but I was actually incredibly surprised by the results. I think the test absolutely nailed me, and I&#39;m so impressed by the accuracy of my test results that I think it&#39;s worth sharing here. These strengths are ranked from strongest to less-strong (I don&#39;t say weakest because it&#39;s only the top 5 of all 34 strengths, so all 5 are very strong).</p> <h1>Strengths</h1> <h2>#1: Analytical</h2> <blockquote> <p>People exceptionally talented in the Analytical theme search for reasons and causes. They have the ability to think about all the factors that might affect a situation.</p> </blockquote> <p>I&#39;m not sure I agree that this is my #1 strength, but it&#39;s definitely very accurate for me. I think this goes hand in hand with the fact that I love being a programmer, and I enjoy debugging code. I&#39;ve never been satisfied with any explanation that something &quot;just is&quot; - I have to understand why something happened or I can&#39;t relax about it. It&#39;s not enough for a stressful production outage to be over, I need to get to the root cause of it. This is true even in the rare instance that there&#39;s nothing I can do about it, and thus get no value out of knowing the cause - the knowledge is the reward for me.</p> <p>I have a tendency to demand people &quot;prove it&quot; when making claims, even believable ones. Similarly, I expect other people to hold me to the same standard - I actually enjoy when I have an explanation for something and friends or co-workers are able to shoot it down. I want to believe as many true things as possible, and disbelieve as many false ones.</p> <p>Nothing is ever noise for me - there&#39;s always some kind of pattern that I want to find in the noise.</p> <h2>#2: Deliberative</h2> <blockquote> <p>People exceptionally talented in the Deliberative theme are best described by the serious care they take in making decisions or choices. They anticipate obstacles.</p> </blockquote> <p>Yep, this is absolutely me. This &quot;strength&quot; is so strong within me that it can often be a weakness, one that I actively try to overcome regularly - I often can get &quot;analysis paralysis&quot;. </p> <p>It takes a great deal of information gathering before I&#39;m willing to make any big decision - I ask an annoyingly large number of questions. This is true even for day-to-day things; the number of questions I asked my realtor when buying my first house actually caused him to become so exasperated that he and I had to part ways.</p> <p>When facing any decision, the first thing I want to know is what the risks are, and I try to plan for every possible outcome. If I don&#39;t feel like I understand the risks of a decision, I often cannot make one.</p> <p>One of the ways I try to address the analysis paralysis weakness is by plowing forward with functional spikes and throwaway code experiments, that way I&#39;m not just stuck reading wikipedia pages or StackOverflow posts. Experimentation is often the best way to acquire real knowledge.</p> <h2>#3: Learner</h2> <blockquote> <p>People exceptionally talented in the Learner theme have a great desire to learn and want to continuously improve. The process of learning, rather than the outcome, excites them.</p> </blockquote> <p>Kind of surprised this one wasn&#39;t higher, though 3 out of 34 is still pretty high. I&#39;ve been accused of being a &quot;perpetual student&quot; on more than one occasion, which is fair since I&#39;ve only been able to last one year after graduating before wanting to go back to school for another degree.</p> <p>I regularly take MOOC classes, attend conferences, and watch talks online. The level of excitement I felt when the company I work for announced that engineers could get free accounts on O&#39;Reilly Safari and Pluralsight is, frankly, embarassing. I read nonfiction constantly, usually jumping between 3 or 4 books at a time, and I consider reading fiction a waste of time because I don&#39;t learn anything while reading, and that time could be better spent learning.</p> <p>One of the things that Gallup says I should try to focus on as an action item as a Learner is:</p> <blockquote> <p>Seek roles that require some form of technical competence. You will enjoy the process of acquiring and maintaining this competence.</p> </blockquote> <p>A-yup, the fact that it&#39;s basically impossible to ever feel &quot;caught up&quot; in this industry is one of my favorite things about it. There&#39;s always stuff to learn, and I love mastering new skills, languages, and technologies.</p> <h2>#4: Intellection</h2> <blockquote> <p>People exceptionally talented in the Intellection theme are characterized by their intellectual activity. They are introspective and appreciate intellectual discussions.</p> </blockquote> <p>Very accurate. One of my favorite pastimes is discussing totally unimportant nerd shit with friends for hours and hours. </p> <p>I like exercising my brain muscles, solving problems, and challenging myself. When I was narrowing down college choices, eventually the deciding factor between my last two options was that I simply wanted to go to the school that I thought would be harder (the one that didn&#39;t offer me a full ride, so I&#39;m still paying for this strength).</p> <p>I&#39;ve often changed my opinions on issues because I challenged myself on some of my beliefs, and tried to reason my way from first principles to a new conclusion, and found myself on the oppsosite side of an issue than my gut reaction was initially.</p> <p>I don&#39;t think the test is telling me I&#39;m &quot;smart&quot;, which would definitely be horoscope territory. But I think you can be dumb and still really enjoy thinking, so I think this description is still fair. </p> <h2>#5: Restorative</h2> <blockquote> <p>People exceptionally talented in the Restorative theme are adept at dealing with problems. They are good at figuring out what is wrong and resolving it.</p> </blockquote> <p>I think this is, in a lot of ways, a natural result of some of the other strengths. As I said before, I love debugging and solving problems. I have a co-worker who is fond of saying that he wouldn&#39;t want to be a murderer if I was the detective assigned to the case.</p> <p>Some of my &quot;action items&quot; for this strength are particularly entertaining to me.</p> <blockquote> <p>Seek roles in which you are paid to solve problems. You might particularly enjoy roles in medicine, consulting, <strong>computer programming</strong>, or customer service, in which your success depends on your ability to restore and resolve.</p> </blockquote> <p>Uh, yeah.</p> <h1>Weaknesses</h1> <p>After taking a class with the guy at work is so gung-ho for this test, he released my full ranking of all 34 strengths. This means I am able to see what my bottom 5 strengths (weaknesses) are.</p> <p>When I saw my bottom 5 weaknesses, I was very happy to discover that they came out of that pool of 10 or so strengths where I&#39;d have dismissed the entire test if it had told me I was strong with one of them.</p> <p>So here they are, in reverse order, meaning starting with my biggest weakness.</p> <h2>#1: Developer</h2> <p>The name of this strength fooled me at first, I was extremely annoyed the test said I shouldn&#39;t be a Developer. But in fact, this is the description:</p> <blockquote> <p>People who are especially talented in the Developer theme recognize and cultivate the potential in others. They spot the signs of each small improvement and derive satisfaction from these improvements.</p> </blockquote> <p>Oh, yeah, that&#39;s me, fine. Really not big on cultivating the potential in others. I won&#39;t try to beat someone else down or something, but I&#39;m much more interested in cultivating the potential in myself than some other bozo. Executive material I am not.</p> <h2>#2: Positivity</h2> <blockquote> <p>People who are especially talented in the Positivity theme have an enthusiasm that is contagious. They are upbeat and can get others excited about what they are going to do.</p> </blockquote> <p>Hah, yep. Not me. I&#39;m almost bummed that &quot;Negativity&quot; isn&#39;t one of the 34 strengths - I guess in a way the Deliberative strength is that - it&#39;s hard to be positive when your main thing trying to figure out the risks of every possible decision.</p> <h2>#3: Includer</h2> <blockquote> <p>People who are especially talented in the Includer theme are accepting of others. They show awareness of those who feel left out, and make an effort to include them.</p> </blockquote> <p>Guilty as charged. I don&#39;t exclude people, but I definitely don&#39;t go out of my way to include anyone. I treat everyone pretty much the same (&#39;Consistency&#39; is my #8) and don&#39;t go out of my way to give anyone special treatment.</p> <h2>#4: Woo</h2> <blockquote> <p>People who are especially talented in the Woo theme love the challenge of meeting new people and winning them over. They derive satisfaction from breaking the ice and making a connection with another person.</p> </blockquote> <p>How is this not my bottom one? I&#39;d be downright offended if this test told me I was good at this. The notion of &quot;winning people over&quot; is about as far from me as you could get.</p> <h2>#5: Empathy</h2> <blockquote> <p>People who are especially talented in the Empathy theme can sense the feelings of other people by imagining themselves in others’ lives or others’ situations.</p> </blockquote> <p>Alright Gallup, I get it. I&#39;m misanthropic as hell. In my defense, I think I&#39;m more personable and enjoyable to be around than a lot of engineers I&#39;ve met. </p> <h1>Conclusions</h1> <p>I really liked this test, and I think it definitely nailed me. Basically the test tells me that I&#39;m doing exactly what I ought to be doing in terms of career choices, which is nice.</p> <p>The book that surrounds this test (and the BigWigs who introduced it at work) put a lot of stock into these results. We&#39;re actually supposed to include our Top 5 in our e-mail signatures, and everyone is supposed to cater how they interact with people to the recipient&#39;s strengths. I&#39;m not entirely sure how I would do such a thing, like what would it mean if I was talking to someone who&#39;s really good at Empathy? Should I talk not about the facts of a task, but of how it makes me feel so they empathize? Weird, pass.</p> <p>One of the things the book and surrounding material pushes though, is playing to your strengths rather than trying to cover your weaknesses. This advice makes sense to me - essentially you want to emphasize the areas where you&#39;re strongest, so I should embrace the fact that I&#39;m an analytical, deliberative, learning, thinking problem-solver and try to make sure that, in my day-to-day work, I&#39;m giving those strengths a chance to shine.</p> <p>There&#39;s also value in knowing your true &quot;lesser strengths&quot;, like the things I got ranked #6-15 for example. Those are truly areas where I am strong, but could be stronger, and it&#39;s worth investing in improvement. My bottom 5, on the other hand, are probably permanently out of reach, so there&#39;s no point in putting myself into a situation where I need to improve my Woo skills, as an example.</p> <p>My #6-15 strengths are:</p> <ul> <li>Futuristic (inspired by the future and what could be)</li> <li>Relator (enjoy close relationships with others)</li> <li>Consistency (treat people the same)</li> <li>Input (like to collect and archive information)</li> <li>Strategic (create alternative ways to succeed)</li> <li>Focus (follow through and stay on track)</li> <li>Achiever (hard worker, likes staying busy)</li> <li>Command (take control of a situation and make decisions)</li> <li>Responsibility (take ownership of what you say you&#39;ll do)</li> <li>Discipline (enjoy routine and structure)</li> </ul> <p>I think this assessment is very fair as well. All ten of those things are things I&#39;m good at, but I could improve, and I definitely understand why they&#39;d be ranked lower than my more-dominant top 5 strengths. <strong>These are all things that there&#39;s value in working on, whereas my bottom 5 are basically a lost cause, so it&#39;s an inefficient use of my time and energy to really focus on them.</strong> It&#39;d be better to focus on my top strengths, and improve my almost-as-strong strengths.</p> <p>Some co-workers who took this test disagreed with the results so strongly that they took it multiple times until they got different results. That makes it even harder to want to actually cater my interactions with people to <em>their</em> strengths. But for me, I think this test absolutely got my number and I think it&#39;s worth doing. The book that goes with this test, <a href="http://www.amazon.com/Strengths-Based-Leadership-Leaders-People/dp/1595620257">Strengths Based Leadership</a>, costs less than $20 and comes with an access code to take the test. I&#39;ve only skimmed the book so I can&#39;t speak to it&#39;s quality, but I definitely recommend taking the test.</p> Sat, 30 Apr 2016 00:00:00 -0400 http://www.nomachetejuggling.com/2016/04/30/strengthsfinder/ http://www.nomachetejuggling.com/2016/04/30/strengthsfinder/ Star Wars Machete Order: Update and FAQ <p>Wow, this <a href="http://www.nomachetejuggling.com/2011/11/11/the-star-wars-saga-suggested-viewing-order/">Machete Order</a> thing got big! After the post first &quot;went viral&quot; and got mentioned on <a href="http://www.wired.com/2012/02/machete-order-star-wars">Wired.com</a>, I started getting around 2,000 visitors to it per day, which I thought was a lot. But then in the months before <em>Star Wars Episode VII: The Force Awakens</em> was released, it blew up like Alderaan, peaking at 50,000 visitors DAILY. This year, over 1.5 million unique users visited the page. <a href="http://www.google.com/trends/explore?hl=en-US&amp;q=machete+order,+cure+for+cancer,+lindsay+lohan+naked&amp;cmpt=q&amp;tz=Etc/GMT%2B5&amp;tz=Etc/GMT%2B5&amp;content=1">It&#39;s been nuts</a>.</p> <p>So let me start out by thanking everyone for liking and spreading the original post - I&#39;m truly floored by how well-received the post was. Considering I wrote a nearly 5,000-word essay on Star Wars, I&#39;m pretty amazed that it was only a handful of times someone told me I was a loser neckbeard who needs to move out of my parents&#39; basement and get a girlfriend (I&#39;m married with a kid by the way). People only called for my public execution a couple times. On the internet, that&#39;s the equivalent of winning an Oscar, so thanks everyone!</p> <table class='image aligncenter' width='646'><tr><td><img src="http://www.nomachetejuggling.com/assets/machete_order_popularity.png" width='640' height='141'/><p class='caption'>Holy shit!</p></td></tr></table> <p>In all seriousness, I&#39;ve had thousands of people tell me I &quot;fixed&quot; Star Wars and made the saga more enjoyable for them. I think this is an unnatural amount of praise - after all, I&#39;m just a guy who watched some movies in the wrong order and skipped one, then wrote down why. I didn&#39;t create fanedits or anything truly difficult like that. But at the same time, the reason I published the post in the first place was that I felt Machete Order &quot;fixed&quot; Star Wars for me personally, allowing me to use the relevant parts of the Prequels to make Return of the Jedi a better movie, so it&#39;s really awesome that so many other people felt similarly. <strong>All joking aside, thank you.</strong></p> <p>Since it&#39;s been about 4 years since the original <a href="http://www.nomachetejuggling.com/2011/11/11/the-star-wars-saga-suggested-viewing-order.html">Machete Order</a> post, and now that Episode VII is out, <strong>I thought I&#39;d post a small update answering a lot of the questions I&#39;ve been asked</strong> and responding to the most common criticisms of Machete Order. <strong>There will be no spoilers of Episode VII here</strong>, though I will be talking about it a bit and I can&#39;t predict what people will post in the comments, so if you haven&#39;t seen it yet, make like a Tauntaun and split.</p> <!--more--> <h1>But Episode I has Maul!</h1> <p><em><strong>&quot;Are you really advocating I never watch Episode I or show it to anyone?&quot;</strong></em></p> <p>Man, no. By far the most common complaint is that I am advocating never watching Episode I, and that&#39;s a shame because it has the best podrace/duel/song/whatever. So let me be perfectly clear, I am not advising anyone to pull their Episode I disc out of their box set and throw it in the garbage. By all means, watch Episode I. Hell, I think Episode I is probably a better movie than Episode II is.</p> <p>The point of Machete Order is not, and has never been, ignoring Episode I because it&#39;s bad. It&#39;s been about skipping it because it&#39;s not relevant to Luke&#39;s journey. Episodes II and III are, because we see how his father falls to the Dark Side, and we see elements of his path that are mirroring his father&#39;s. </p> <table class='image alignright' width='306'><tr><td><img src="http://www.nomachetejuggling.com/assets/fates.jpg" width='300' height='169'/></td></tr></table> <p><strong>By all means, if you like Episode I, watch it.</strong> What I&#39;m advocating though, is watching it sort of like an Anthology film - remember that we&#39;re going to be getting Han Solo origin movies and Boba Fett spinoffs and Rogue One films, and so on, until Disney stops making money off Star Wars. These movies are all going to take place at different times, between different Episodes, or before all of them. If you enjoy or want to share Episode I, I say view it as an Anthology movie, sort of like a prequel to the entire series.</p> <p>In other words, when you&#39;re watching &quot;The Main Saga&quot;, like maybe if you&#39;re doing a Marathon or you&#39;re introducing someone to Star Wars for the first time, watch in Machete Order: IV, V, II, III, VI. When you&#39;re done and that &quot;book&quot; is closed, you can pull in whatever &quot;Anthology&quot; stuff you enjoy, such as the Clone Wars TV shows or movies, the Han Solo spinoff, and Episode I. </p> <p>But for some kind of contiguous viewing experience, I think Episode I should be skipped, because it provides mostly backstory to the Republic itself and political goings-on. This makes it an interesting prequel to the entire saga, but a useless distraction from Luke&#39;s journey. </p> <h1>But Episode I has backstory!</h1> <p><em><strong>&quot;Aren&#39;t parts of Episode I crucial pieces to the story?&quot;</strong></em></p> <p>No, they aren&#39;t. They might be crucial pieces to the Star Wars overall story, but not to Luke&#39;s story, which is the whole point of Machete Order: re-centering the main saga narratively on Luke.</p> <p>Yes Sheldon, <a href="http://www.youtube.com/watch?v=keSFjjhUyVA">Chancellor Valorum is relevant</a> to understanding Palpatine&#39;s rise to power. Yes, Qui-Gon&#39;s belief that Anakin is the chosen one, combined with his untimely demise are very directly responsible to understanding Anakin&#39;s fall. Those make them interesting backstory - but they are <strong>not relevant to Luke&#39;s journey</strong>.</p> <table class='image alignleft' width='306'><tr><td><img src="http://www.nomachetejuggling.com/assets/amidalabeforesenate.jpg" width='300' height='153'/></td></tr></table> <p>People who point this out act like it&#39;s sacrilege to (temporarily, see above!) skip Episode I because it fleshes out the Star Wars universe in various ways. So they might advocate Episodes I, II, III, IV, V, VI, VII, in order. But imagine that Disney releases an Episode 0, all about how Qui-Gon ignored some other ancient Jedi prophecy, and as a result his entire family died or something. This would provide a great understanding of why Qui-Gon is so insistent on training Anakin, and why he passes that burden to Obi-Wan. If someone were to suggest skipping Episode 0, by the logic of Machete Order detractors this would be impossible, because it&#39;s critical in understanding Qui-Gon&#39;s motivations. But skipping it would simply be regular Episode Order that we have now, which is what they&#39;re arguing for. This could go back forever, the exact order being advocated as &quot;correct&quot; is somehow now missing a critical component, because it skips hypothetical &quot;Episode -1&quot; and &quot;Episode -2&quot;.</p> <p>In other words, we don&#39;t really need to know why Qui-Gon is so intent on Anakin being trained or why he believes so strongly in a prophecy that the rest of the council doesn&#39;t seem to care much about. &quot;He just does&quot; is a perfectly fine answer for now, and it would be a perfectly fine answer if Episode 0 existed too. Similarly, we don&#39;t really need to know all of the machinations that led to Anakin embracing to the dark side, &quot;he just does&quot; is perfectly suitable, and in fact I argue that &quot;he lacks proper training&quot; is a far less sympathetic answer than &quot;it&#39;s very seductive&quot;, which is what we&#39;re left with skipping Episode I.</p> <p>All of these movies make references to past events that we don&#39;t ever see on screen. That&#39;s what these big &quot;worldbuilding&quot; movies are all about, and why there&#39;s a whole business for books and comics and video games to support them. We don&#39;t <em>need</em> to see Anakin&#39;s mother becoming a slave (not even in a movie), just like we don&#39;t <em>need</em> to know exactly why Nute Gunray hates Padme so much in Episode II. It&#39;s all backstory and fleshes things out a bit, but it&#39;s not critical, your mind fills in the gaps, makes educated guesses, and so on.</p> <p>Bear in mind, people happily enjoyed Star Wars without ANY of the prequels for sixteen years, and nothing that happened in the original trilogy left some kind of gaping unanswered question in the minds of the audience. So really, since the whole point of Machete Order is refocusing the story on Luke, claiming that any part of the prequels is truly <strong>necessary</strong> is a bit of a hard sell. I argue that Episodes II and III make Luke&#39;s story more enjoyable to watch in VI, but <em>crucial</em>? As in, unable to be understood without them? Nah.</p> <h1>But the prequels aren&#39;t that bad!</h1> <p><em><strong>&quot;I grew up with the prequels and they&#39;re not as bad as you think! You&#39;re blinded by nostalgia for the originals!&quot;</strong></em></p> <p>I had no idea what a huge population there was of Prequel fans, people who were born in the 90&#39;s and grew up watching the prequel trilogy and love them. Many people even claim Episode I is their favorite, or their favorite character is Jar-Jar. These people are not trolls, they genuinely love these movies. In fact they claim that the only reason that myself and others dislike the prequels is because our own nostalgia for the original trilogy blinds us to their flaws.</p> <p>First, a bit of an admission: I am not a huge Star Wars <a href="https://www.washingtonpost.com/lifestyle/in-what-order-should-you-watch-the-star-wars-movies/2015/12/09/25e96e88-9cf8-11e5-a3c5-c77f2cc5a43c_story.html">&quot;superfan&quot;</a>; I&#39;m just a movie geek. If I was some kind of rabid Star Wars fanboy, I would imagine I&#39;d consider it borderline blasphemous to advocate skipping an entire film in the Gospel of Star Wars. But as a movie nerd, I&#39;m more than happy to make whatever adjustments I think make for a better film-watching experience, because Star Wars is just a bunch of movies to me. I skip Godfather III and The Incredible Hulk too. They&#39;re just movies.</p> <p>So, here&#39;s my big secret: <em>I did not grow up watching Star Wars</em>. In fact, whenever I saw clips or images from the movies, I thought they looked boring (it looked like they mostly took place in the desert), and I skipped them. I liked parody movies, so I watched Spaceballs instead (a bunch). It was not until I was a senior in high school that my older sister discovered I still hadn&#39;t seen any Star Wars movies, and insisted I watch them. This was in 1999. To reiterate: <strong>I saw Episodes IV, V, VI, and I all for the first time, the same year, when I was seventeen.</strong></p> <table class='image alignright' width='306'><tr><td><img src="http://www.nomachetejuggling.com/assets/spaceballs.jpg" width='300' height='169'/><p class='caption'>My Star Wars</p></td></tr></table> <p>As a result, I can confidently say that I am not blinded by nostalgia for the original trilogy - they played no role in my childhood. I saw Episode I almost immediately after seeing the original trilogy, and I feel totally justified in saying that the prequel trilogy films, every single one of them, is vastly inferior to the original trilogy entries. I think my opinion here is pretty much objective - in fact I think the younger crowd talking about the greatness of the prequels are the ones blinded by their nostalgia.</p> <p>Further, the very first versions of the original trilogy I saw were the Special Editions, because that&#39;s what was available on VHS at the local video store at the time. Han never shot first for me. A cartoon Jabba always talked to Han after Greedo, Jabba&#39;s palace has always had an extended dance number, and the entire galaxy (not just Ewoks) always celebrated the fall of the Empire, at least for me. I didn&#39;t see the &quot;Despecialized&quot; versions until years and years later, and so I can once again confidently say, with total objectivity, that they are better than the special editions. The improved special effects for Cloud City and some matte improvements are welcome, but otherwise the Special Editions make the movies worse.</p> <p>Look, you can like or even love the prequels, and I totally understand why you might if you grew up watching them. But really, they are dreadfully bad movies, as far as movies go. Frankly I also think Return of the Jedi isn&#39;t a very good movie either, it&#39;s a mediocre movie that&#39;s elevated by having stellar <em>moments</em>. But all three of them are parsecs better than all of the prequels (yes, even III, &quot;the good one&quot;).</p> <p>It doesn&#39;t make the prequels genuinely good movies just because you liked them when you were a kid. Kids are completely capable of loving terrible movies. Kids are stupid. When I was a kid, I thought the two best movies in the world were Back to the Future and Superman III. Turns out, one of them is genuinely good, and one of them is actually dog shit.</p> <p>I am officially completely dismissing outright any criticism that my dislike for the prequels is because of my nostalgic childhood affection for the originals. I have no such childhood affection, and the prequels are dreck. Sorry.</p> <h1>What About Force Lightning?</h1> <p><em><strong>&quot;Doesn&#39;t Machete Order ruin the surprise that Emperor Palpatine can shoot lightning?&quot;</strong></em></p> <p>Yep, sure does. This was something I hadn&#39;t realized before, and was pointed out to me by a commenter. But indeed, if you&#39;re watching the original trilogy, the first time Palpatine starts electrocuting Luke, it&#39;s quite a shock (har har). </p> <p>With Machete Order, this surprise happens when Count Dooku just casually does it in Episode II. It&#39;s a real shame because it doesn&#39;t have the emotional or narrative impact here. I have no real defense for this, and I actually now consider it Machete Order&#39;s greatest flaw.</p> <table class='image alignleft' width='306'><tr><td><img src="http://www.nomachetejuggling.com/assets/forcelightning.jpg" width='300' height='155'/></td></tr></table> <p>I kind of always thought the lightning wasn&#39;t a &quot;Sith power&quot; so much as something that Palpatine could do because he&#39;s so incredibly fucking evil. But no, the prequels make it clear this is just one of the video game powers you get by embracing the darkside, and they just do it willy nilly all over the place. Apparently you can just absorb it with a lightsaber if you have one handy, or without one if you&#39;re Yoda (hint to Luke, don&#39;t throw your lightsaber away, it has a +2 against Force Lightning!)</p> <p>It&#39;s even kind of annoying that this is typically referred to as &quot;force lightning&quot; now, like it&#39;s some kind of standard-issue thing you learn in Graduate Level Sith Academy before you get your diploma. I think it was better when it was just &quot;that evil scary crazy lightning shit The Emperor does out of nowhere.&quot; But alas, the prequels ruined this (have I mentioned that they suck?) and Machete Order is unable to fix it. </p> <p>The only way to preserve this twist is to simply move Episode VI two movies earlier, which is effectively just Release Order (IV, V, VI, I, II, III). I like the lightning surprise a lot but I think overall it&#39;s worth giving it up in order to make the final confrontation between the Emperor, Vader, and Luke more enjoyable by watching II and III first.</p> <p>The best defense I can offer is that there&#39;s basically no way to preserve this twist without moving the &quot;Luke and Leia are twins&quot; surprise back to Episode VI. And as I&#39;ve pointed out elsewhere, it actually works far better at the end of III, when the audience has no idea they are related, but does know who they are (by watching IV and V before it). So in a sense, you kind of have to choose if you want an effective twin twist or an effective lightning twist, and I personally choose the twins.</p> <h1>Where Does Episode VII fit?</h1> <p><em><strong>&quot;Now that The Force Awakens is out, how does it fit in Machete Order?&quot;</strong></em></p> <p>It&#39;s amazing how often I&#39;ve been asked this. I was even getting asked this question before I&#39;d seen the movie, in the form of &quot;now that (spoiler), where does it fit with Machete Order!?&quot; so thanks for that.</p> <p>But I could have answered this question before even seeing Episode VII, and now that I&#39;ve seen it I&#39;m just as confident in my answer. <strong>It goes after</strong>. </p> <p>IV, V, II, III, VI, VII.</p> <p>The Force Awakens is a continuation of the story, and without giving any spoilers away, having now seen the film I&#39;m confident the entire sequel trilogy is going to be viewable as a continuation or epilogue to Luke&#39;s Journey. Thus it is even more appropriate than ever to refocus the first part of the saga, Luke&#39;s adventure as a young man, by framing Episodes II and III within the narrative as relevant to Luke&#39;s own temptations. </p> <p>Clone Wars, Episode I, Rogue Squadron, Young Han Solo, The Adventures of Yoda, Boba Fett&#39;s Big Day, and whatever other movies come out that aren&#39;t part of the numerical &quot;Episode&quot; structure, can all pretty much be viewed in any order, as supplemental material outside of the &quot;main saga&quot; which I suggest viewing in the above order. Before watching any of those movies, you just say &quot;this takes place between X and Y&quot; and you&#39;re done. I don&#39;t imagine Episode 7, or even Episode 9 will be the last Star Wars Episode that Disney releases, either. It&#39;s entirely possible that the Episodes 10-12 &quot;trilogy&quot; will be a completely separate thing, but for now it seems clear that going forward my position is going to be:</p> <p>IV, V, II, III, VI, VII, VIII, IX, X, XI, XII...</p> <table class='image alignright' width='306'><tr><td><img src="http://www.nomachetejuggling.com/assets/lukemad.jpg" width='300' height='200'/></td></tr></table> <p>Now, all of that being said, while watching Episode VII I was struck by the way that certain parts and characters of the original trilogy had become &quot;legend&quot;. The movie&#39;s final shot is pretty big tie-in to one of those legends, so I think another interesting Star Wars order would be (contingent upon what happens in VIII and IX): 7, 4, 5, 6, 8, 9. I wouldn&#39;t bother inserting the prequels at all there. Again, who knows what&#39;s in store, but the way a particular character is framed throughout The Force Awakens I have a hunch that order might be fun someday.</p> <p>In any case, for the time being my answer is: watch Star Wars in Machete Order (4, 5, 2, 3, 6), and then watch all subsequent episodes in the order they&#39;re released, because they are both in episode order and chronological order, so there&#39;s no reason to play musical chairs with them. </p> <h1>But What If The Next One...?</h1> <p><em><strong>&quot;What if Episode VIII reveals that (spoiler)?&quot;</strong></em></p> <p>There&#39;s lots of speculation in the wake of Episode VII&#39;s release. There&#39;s a <a href="http://moviepilot.com/posts/3691694">particular piece of speculation</a> that I personally am not dismissing at all, which would present a bit of a problem for Machete Order if it turns out to be true. Basically, if any of these Star Wars movies come out and have something happen that will basically make <em>no sense</em> unless you&#39;ve seen Episode I, Machete Order is bunk. But I think that&#39;s very unlikely to happen, Disney is trying to capture a new audience, even Episode VII is largely watchable even if you&#39;ve never seen any of the previous movies. Hell, any one of the Star Wars films works without any other Episodes - the whole point of the opening crawls is that they harken back to the days of Flash Gordon and Buck Rogers serials where people would permanently miss episodes and need summaries to be caught up.</p> <table class='image alignleft' width='306'><tr><td><img src="http://www.nomachetejuggling.com/assets/serkis.jpg" width='300' height='186'/></td></tr></table> <p>Is it possible that Episodes VIII or IX would contain some revelations or twists that would be more enjoyable if you&#39;ve seen Episode I? Yeah, completely possible, but I doubt anything will make Episode I required viewing. So I&#39;m still confident in saying, even if you&#39;re watching Star Wars films in preparation to watch entries in the sequel trilogy, Episode I can still be skipped.</p> <p>Besides, <a href="http://www.ew.com/article/2015/11/30/jar-jar-binks-fan-theory-jj-abrams">&quot;Darth Jar Jar&quot; was dismissed by J.J. Abrams</a> so <strong>I think Machete Order is safe</strong>.</p> <h1>Did you like Episode VII?</h1> <p><em><strong>&quot;What did you think of The Force Awakens?&quot;</strong></em></p> <p>I&#39;m surprised people care what I think of the movie itself. I may publish a full review on another movie blog I run, but let me summarize with: <strong>it&#39;s fine</strong>. I think it&#39;s generally pretty forgettable like most of J.J. Abrams&#39;s movies, and I think like all his other movies, the more you think about what happens in it, the less sense it makes.</p> <p>But I&#39;m also reserving judgment until the other episodes are released. Lots of things that seem like huge coincidences or plot holes could easily be addressed and explained in subsequent films. It&#39;s certainly the best movie J.J. has ever made, and it&#39;s definitely better than the prequels, at the very least in terms of technicals.</p> <p>Aesthetically and tonally, it&#39;s much more similar to the original trilogy than it is to the prequel trilogy, and I&#39;d argue far closer to the originals than the prequels were themselves. It &quot;feels like Star Wars&quot; though I think a big part of that is that it leans so heavily on the original films that one could easily mount an argument that it&#39;s a soft reboot. I&#39;m certainly not the first to point out how much it borrows from the prior films, and I think it&#39;s a legitimate criticism.</p> <table class='image alignright' width='306'><tr><td><img src="http://www.nomachetejuggling.com/assets/kyloren.jpg" width='300' height='188'/></td></tr></table> <p>In a lot of ways, Episode VII is a highly cynical piece of marketed, productized material. For all their faults, you can tell that the prequel trilogy films existed because George Lucas had a story that he really wanted to tell. Episode VII didn&#39;t give me that feeling, it didn&#39;t come off like a movie that was dying to get out of the creative mind of an auteur. It felt like a product that had been assembled and expertly directed by a committee of very smart businesspeople. It was naked and transparent in its intentions and its methodologies, pulling the strings to elicit an almost impossible-to-avoid sense of joy in the audience.</p> <p>None of this is to say it&#39;s bad - far from it. I, the puppet, was able to look up and see all of the strings attached to me, and notice as they were pulled and prodded. I rolled my eyes, knowing exactly what my puppet masters were doing and why. But it didn&#39;t change the fact that the puppet masters were <em>successful</em> in eliciting the desired movements from me. <strong>They forced me to feel joy, I was aware that it was being forced, but I felt it nonetheless</strong>. </p> <p>I enjoyed the hell out of Episode VII and I consider it an extremely good but flawed movie, a quality &quot;Star Wars Sequel&quot; like Return of the Jedi, that manages to be a good franchise entry but not an &quot;excellent film&quot; on its own in the way that Empire Strikes Back and the original Star Wars are. Like Die Hard is a classic piece of near-perfect filmmaking, it belongs in the pantheon of great films for all time. Die Hard 3 is a really good Die Hard sequel, but it&#39;s not a timeless classic like Die Hard. Same deal for Return of the Jedi and The Force Awakens - great entries in their franchises, but not Top 100 Films of all Time material.</p> <p>In any case, I&#39;m definitely looking forward to Episode VIII. I think the franchise is in good hands and, made-by-committee or not, there&#39;s a level of consistent quality that makes me confident in future installments. I hope that Episode VII got all of the placating fanservice out of the way and calmed everyone&#39;s fears about the upcoming slate of movies, so now they can focus on just telling interesting, original stories set in the Star Wars universe, and I think that&#39;s relatively likely (fingers crossed).</p> <h1>Other Stuff</h1> <p>Those are all the questions I get regularly. I think I&#39;ll update this one post with new questions I get in the future, so that my poor little Software Engineering blog doesn&#39;t turn into Star Wars Central or something. If you have other criticisms of Machete Order or other questions, feel free to leave a comment. I&#39;ve gotten over 1,000 comments on the original post, and I read them all.</p> <p>And again, thank you to everyone who made Machete Order blow up all over the place. I&#39;ve been on the radio multiple times and <a href="http://www.npr.org/2014/03/20/291977042/theres-more-than-one-way-to-watch-star-wars">NPR</a>, and had articles that mention me by name published in <a href="http://www.nydailynews.com/entertainment/movies/star-wars-fans-debate-movie-marathon-viewing-order-article-1.2454281">New York Daily News</a>, <a href="https://www.washingtonpost.com/lifestyle/in-what-order-should-you-watch-the-star-wars-movies/2015/12/09/25e96e88-9cf8-11e5-a3c5-c77f2cc5a43c_story.html">Washington Post</a>, and <a href="http://www.cnn.com/2015/12/08/entertainment/star-wars-machete-order/">CNN</a>. The order has been mentioned on <a href="https://www.youtube.com/watch?v=effD1u4oCRE">King of the Nerds</a>, <a href="https://www.youtube.com/watch?v=keSFjjhUyVA">The Big Bang Theory</a> and <a href="https://www.youtube.com/watch?v=XP0F1eKJZ3s">Late Night with Seth Meyers</a> by one of my favorite comedians, Patton Oswalt. As far as 15 minutes of fame go, it&#39;s been a real blast, and I have everyone who saw the post and shared it to thank.</p> <p>May the Force be with you, always.</p> Mon, 28 Dec 2015 00:00:00 -0500 http://www.nomachetejuggling.com/2015/12/28/machete-order-update-and-faq/ http://www.nomachetejuggling.com/2015/12/28/machete-order-update-and-faq/ PhD Status Report <p>It&#39;s been a long time since I posted about how school is going, and I figured the folks who read my blog (all two of you, hi Mom!) might be curious.</p> <p>Since the end of my Spring 2013 semester, I&#39;ve been in &quot;Research Phase&quot;. This means I&#39;m finished with classwork and have been working on my research project. It&#39;s been a little over two years now, so here&#39;s what has happened.</p> <h1>Picking a Project</h1> <p>About two years ago, I started working with my advisor trying to figure out what area my research would be in. I&#39;ve always had a fascination with Genetic Algorithms and Metaheuristics, so I knew I wanted to do something involving that subfield. Two of my projects at school utilized Genetic Algorithms, and I had a lot of experience in the area.</p> <p>I went back to school to primarily focus on CS Theory &amp; Algorithms, but I knew my biggest strength was Software Engineering. In other words, I like theory, but I&#39;m not sure I&#39;m a real theoretician, and I wanted to play to my strengths a bit. This meant I wanted to be writing real, working code, rather than proofs. If I wound up proving anything interesting, that&#39;d be great, but I didn&#39;t want that to be part of my critical path to completing my PhD. Lots of CS PhD&#39;s go off with a white board and paper and output some amazing results, but I knew that wasn&#39;t going to be me.</p> <p>My first idea was to build some kind of framework for comparing different metaheuristics. I&#39;ve worked extensively with Genetic Algorithms but that&#39;s just one of an entire class of Metaheuristics. I had no experience with things like Swarm algorithms, Ant Colony System optimization, Tabu search, Simulated Anealing, and so on. I thought it would be interesting to pick some known optimization problems like Traveling Salesman using <a href="http://comopt.ifi.uni-heidelberg.de/software/TSPLIB95/">TSPLIB</a>, code various &#39;solvers&#39; using different techniques, and compare their performances.</p> <p>I started building out the project, codenaming it <code>judy</code> (because it was like a judge, ha ha. My advisor hated this name and told me to change it.) Unfortunately, while doing a literature review in this area I discovered that it&#39;s more or less accepted that all metaheuristics have a similar level of effectiveness.</p> <p>The more I researched, the more I realized that this would probably not be a valuable contribution to the the field. It was becoming very clear that this project wasn&#39;t well-formulated, and wasn&#39;t going to be successful. I hadn&#39;t spent very long on it, building out some parts of Judy but not much more. I needed to pick a different project, but I was still very interested in Metaheuristics, so I knew I wanted it to be closely related.</p> <h1>Course Correction: Picking a Similar But Different Project</h1> <p>I had already built out some parts of Judy by this point and had some pretty promising software. In fact the <a href="https://github.com/rodhilton/metaheuristics-engine">metaheuristics-engine</a> I had been building was a pretty nifty generic simulator for metaheuristic algorithms. I&#39;d used my skills as a software engineer to build a few interfaces that were easy to implement and then dump directly into a simulator that would scale out for however many processors were on a machine, handle as much of the metaheuristic algorithm in parallel as possible, and do various neat things like safe-journaling results (so that an interrupted simulation could resume). Thus, writing new metaheuristic implementations to search problem spaces would be really easy and fun.</p> <p>I wanted to use this software because I enjoyed building it, and I thought it would be cool to enhance the capabilities of my metaheuristic engine as a side-artifact of my research. I had also started building out a taxonomy of metaheuristics and how they were related in terms of object-oriented relationships (for example, a <code>GeneticAlgorithm</code> was a type of <code>EvolutionaryAlgorithm</code> in a formally-defined way, because one extended the other polymorphically).</p> <p>I went to another professor, the one who taught the classes where my projects utilized Genetic Algorithms. I knew she had a whole bag of interesting problems to solve, and I wondered if I could use my simulator to find solutions.</p> <p>Basically all of her problems were existence questions. Things like, does there exist a graph $$G$$ such that $$Y$$, with $$Y$$ being a variety of different restrictions. She&#39;d posed a similar problem before, which was the motivator behind <a href="https://speakerdeck.com/rodhilton/rectangle-visibility-and-elusive-k23">my project in her Graph Theory class</a>.</p> <p>Most of these graphs would have their existence proven not by formal proof, but by construction. Meaning, if I could construct a particular kind of graph, that graph itself would be the interesting result (and not necessarily how I derived it). My idea was, for each of her problems, I would figure out a way to randomly generate potential solutions, a way to score solutions in terms of how close they are to the ideal, and a way to mutate or perform crossover mutation with candidate solutions. I&#39;d write code for the problems, use/enhance my simulator, and then run the simulation on our ultrapowerful university computer across hundreds of threads.</p> <p>When I took on this project, <strong>I knew that it was high-risk/high-reward</strong>. For every problem I was working on, it was entirely possible that the solution I was looking for does not exist (remember, the only proofs possible are via construction). I also knew it was possible that even if it exists, my strategy might not find it, or that my approach might get stuck in local optima. In other words - I was essentially applying a technique intended for discovering SUBOPTIMAL results to optimization problems, hoping that that the threshold for optimality would be low enough that the technique could work.</p> <p>It&#39;s been two years since I started this project. In that time, I have been working on three different particular problems, growing my metaheuristics solution engine, and heavily taxing the resources on the university parallel computation platform.</p> <p>The first problem I worked on has been running for the longest time. Since this was a continuation of my project in my advisor&#39;s Graph Theory class, I simply adapted it to used my new and improved simulator, and put it up on our university multiprocessor machine. I&#39;m not going to bore you with the details but basically I&#39;ve been trying to find a particular graph configuration with certain restrictions, to see if a complete graph of a certain size exists. Since I&#39;m looking for a complete graph, the &#39;fitness&#39; criteria is simply, how many edges my best solution has, out of a possible $$\frac{n(n-1)}{2}$$ edges. About a year ago (after running for a year), my software managed to get ONE edge away from the desired answer. As of this writing, it is still one edge away from the solution I&#39;m trying to discover, which seems to be the best it can do (205,949,225 generations). Unfortunately this is not an interesting or publishable result unless I can uncover that one missing edge. It&#39;s always possible that the software will find the correct solution tomorrow, but I think at this point I need to admit this is a dead-end.</p> <p>I&#39;ve worked on two other problems but of those, I&#39;ve spent about 95% of my time on just one of them. This particular problem (again, I&#39;ll spare the details) is one that&#39;s been plaguing my advisor herself for over twenty years. I vividly recall one day in Algorithms class when she told the class that, if someone can discover particular graph with a specific set of properties, &quot;I&#39;ll give you a PhD right there.&quot; It was this problem I chose to work on for most of these last two years.</p> <p>Again, this too was a high-risk/high-reward project. My advisor and I joked that the entire publication that results from this effort would be a couple pages about the background and the reason for wanting to know if this graph exists, and the entire results section would be a single page - a listing and rendering of the graph. If I had found a result, there&#39;s no question that a paper about it would be accepted in virtually any CS journal I sent it to.</p> <p>But once again, I&#39;ve been working on this project for a little over two years now. I recognize that it seems early to ``give up&#39;&#39; given that it&#39;s been plaguing my professor for decades, but she&#39;s been trying to construct the graph by hand and my goal was to use the metaheuristics to perform a guided search of the solution space computationally. In the end though, I&#39;ve become convinced after two years of working on this that either the solution doesn&#39;t exist, or that it does exist but my technique will not discover it.</p> <h1>Walking Away</h1> <p>This past semester my wife and I found out that we were going to be having a baby girl in April. Very suddenly, I started to evaluate the relative importance of my degree. It started to feel very stupid to be working on a series of high-risk/high-reward research projects where nothing had panned out in two years. These problems were unbounded in every sense - my simulations might discover a solution after a day, a week, a year, ten years, a thousand years, or never. I started to become increasingly uncomfortable with the notion of continuing down a path of research so open-ended, with the possibility that I was programmatically searching for solutions that may not even exist.</p> <p>When I first decided to go back to school, I did it for fun. I enjoyed my Master&#39;s research and wanted to continue learning and educating myself, and I liked the idea of pursuing a Doctorate. At the time, I told myself and my wife that, since I was doing it for fun and not for any professional reason, that I&#39;d stop the instant it stopped being fun. Well, after two years of research that wasn&#39;t panning out, school had certainly stopped being fun for me. In fact, it&#39;s been a source of constant stress and irritation. I&#39;ve been working very hard and building some really cool software, but I have no interesting results to show for it.</p> <p>The analogy I used to describe how I was feeling to my wife was, it was as if I saw people mountain climbing and I thought it looked really fun, and I wanted to do it as well. So I spent a great deal of time and money taking mountainclimbing classes. Then I began researching mountainclimbing gear and spending a great deal of money on equipment. I planned and scheduled flights to various mountain ranges. Then I finally started climbing mountains and realized &quot;you know what? This sucks. I&#39;m really cold, and this isn&#39;t fun at all.&quot; But because I&#39;d spent so much time and money on it, I felt like I had to keep going even though I hated it it, falling victim to the sunk cost fallacy.</p> <p>After a great deal of thought I decided I would walk away from the PhD altogether. This was not an easy decision, given how much work I&#39;d put into things like the GREs, classes (straight A&#39;s), and passing my preliminary exams. But after two years and nothing publishable to show for all my research work, with a baby on the way, I felt like I needed to re-prioritize and know when to fold &#39;em, so to speak.</p> <h1>...Or Not?</h1> <p>I scheduled a meeting with my advisor to discuss my departure from the program. I was all set to walk away, and in fact had written the entire above portion of this blog post. I had basically accepted that I was done, I killed the processes on the university supercomputer, and moved a ton of books and articles out of my to-read list on Goodreads. Washing my hands of the project and admitting I was no longer pursuing it was extremely liberating - I felt like a huge burden had been lifted simply by shutting down the effort. </p> <p>But then I realized something. My main motivator for coming back to school was how much I enjoyed my Master&#39;s Thesis research, and that I wanted to do more. I had thought that I must love research, and wanted to continue it. The past two years made me realize I didn&#39;t actually enjoy research as much as I thought - a fact that I had trouble coalescing with how much I enjoyed my Master&#39;s research.</p> <p>A few weeks ago, the author of <a href="http://smile.amazon.com/Beyond-Legacy-Code-Practices-Software/dp/1680500791?sa-no-redirect=1">Beyond Legacy Code</a> contacted me to say he referenced my Master&#39;s thesis in his book. He praised my effort, talking about how much he enjoyed it and how he felt it was one of the better entries he&#39;s come across. It made me look back at my thesis and remember how much I enjoyed working on it.</p> <p>Then it dawned on me. Was it possible that it wasn&#39;t the research I enjoyed at the time, but the actual subject? If it was the subject, then would a research project more inline with my Master&#39;s appeal to me more? I had come back to school for Theory &amp; Algorithms, and I was so laser-focused on those subjects that it hadn&#39;t occured to me that I might enjoy classes in these topics but <em>not</em> research. Would I enjoy a research project if it were focused on Software Engineering, like my Master&#39;s?</p> <p>Pondering this for a few more days, I suddenly had a new idea for a research project. This was something much more Software Engineering-focused than what I had been working on - in fact it wasn&#39;t related to CS Theory at all. It was more of a natural continuation of my Master&#39;s work, and I was stunned by how <strong>excited</strong> I was by the idea. I once again felt the passion of wanting to complete my PhD, reinvigorated by this new project. My wife commented that she hadn&#39;t seen me this excited about research in the entire two years I&#39;d been working on my metaheuristics project.</p> <p>Of course, there have been some logistical issues. Since I went back to school primarily to focus on Theory, nearly every class I&#39;ve taken has been theory-heavy. Due to this, every single professor I&#39;ve met or worked with has been a theory professor, not a single one of them is experienced, or even interested in Software Engineering research. </p> <p>The meeting with my advisor, which I had originally scheduled to tell her I was dropping out of the program, went very differently. I had come up with the new project and done some publication searching all between scheduling the advising meeting and actually having it. So when I walked into her office, I presented her with my new project idea.</p> <p>I expected her to hate the idea since it&#39;s so unrelated to theory. Instead, her reaction was basically, if I&#39;m passionate about the project she&#39;ll help in any way she can. She doesn&#39;t have a lot of experience with Software Engineering but I don&#39;t think I need a lot of help with the topic - this is what I know best. </p> <p>So that&#39;s my status update. Baby on the way, feeling a strong desire/pressure to finish this PhD quickly, and basically starting over entirely in the research. Reading it back, it seems downright idiotic, but I feel extremely excited about my new project, and for some reason I don&#39;t feel worried.</p> <p>I still feel the same way about the program - this is something I&#39;m doing for fun, and the instant I&#39;m miserable in the program I will walk away. But if there&#39;s any path that ends with me actually completing this degree, I feel like I&#39;m finally on it.</p> <p>The next update about school will either be announcing that I&#39;ve completed my dissertation and am graduating, or that I&#39;ve decided to walk away from the program for good.</p> <h2>TL;DR</h2> <p>Too rambly but you still sort of care? Here it is in a nutshell:</p> <blockquote> <p>I decided the research project I&#39;ve been working on for two years was a dead-end. I almost decided to walk away from the program entirely, but instead have started over with a radically different project to give it one last shot before calling it quits.</p> </blockquote> Fri, 13 Nov 2015 00:00:00 -0500 http://www.nomachetejuggling.com/2015/11/13/phd-research-status-report/ http://www.nomachetejuggling.com/2015/11/13/phd-research-status-report/ Testing Against Template Renders in Grails <p>I work with Grails a lot and while I really enjoy it for the most part, there are definitely some weird quirks of the framework.</p> <p>One such quirk is something I encounter whenever I want to write unit tests against grails controller methods that render out templates directly. This isn&#39;t something I do very often - generally I prefer rendering out JSON and parsing it with client-JS - but in some cases when there&#39;s a lot of markup for a page element that you want to be updateable via ajax, it makes sense to render out a template like <code>render(template: &#39;somePartial&#39;)</code> directly from a controller method.</p> <p>Unfortunately, these kinds of methods are very difficult to write tests against. While a normal render exposes a <code>model</code> and <code>view</code> variable that you can test against, for some reason using a render with a template doesn&#39;t seem to do this.</p> <p>I&#39;ve seen lots of solutions where you stuff a fake string matching the name of the template using some metaclass wizardry, but then you&#39;re stuck dealing with some semi-view stuff in what you might want to simply be a unit test about the model values placed by the controller method.</p> <p>However, based on <a href="http://stackoverflow.com/questions/15141319/grails-controller-test-making-assertions-about-model-when-rendering-a-template">this StackOverflow post</a>, I&#39;ve written a quick-and-dirty little monkeypatch that exposes the <code>model</code> and <code>view</code> variables in your test, and populated with the values relevant to the template.</p> <p>I&#39;ve got this method in a <code>TestUtil</code> class:</p> <div class="highlight"><pre><code class="language-groovy" data-lang="groovy">static def ensureModelForTemplates(con) { def originalMethod = con.metaClass.getMetaMethod('render', [Map] as Class[]) con.metaClass.render = { Map args -&gt; if (args[&quot;template&quot;]) { con.modelAndView = new ModelAndView( args[&quot;template&quot;] as String, args[&quot;model&quot;] as Map ) } originalMethod.invoke(delegate, args) } } </code></pre></div> <p>Then I can call this method with the controller as a parameter anywhere before I invoke the controller action. It can go in a <code>setup</code> or <code>@Before</code> method, it seems to work from both Spock tests and the builtin Grails testing framework.</p> <p>So if we have this example:</p> <div class="highlight"><pre><code class="language-groovy" data-lang="groovy">class ExampleController { def index() { def bigName = params.name.toUpperCase() render(template: &quot;partial&quot;, model: [ one: &quot;hello&quot;, two: bigName ]) } } </code></pre></div> <p>This test will do what we want:</p> <div class="highlight"><pre><code class="language-groovy" data-lang="groovy">@TestFor(ExampleController) class ExampleControllerSpec extends Specification { def setup() { TestUtil.ensureModelForTemplates(controller) } def shouldHaveModelAndViewExposed() { given: params.name = &quot;Rod&quot; when: controller.index() then: view == &quot;partial&quot; model.one == &quot;hello&quot; model.two == &quot;ROD&quot; } } </code></pre></div> Thu, 27 Aug 2015 00:00:00 -0400 http://www.nomachetejuggling.com/2015/08/27/testing-against-template-renders-in-grails/ http://www.nomachetejuggling.com/2015/08/27/testing-against-template-renders-in-grails/ QCon New York 2015: A Review <p>My default yearly conference, for many years, has been <a href="http://uberconf.com/conference/denver/2015/07/home">UberConf</a>. I really enjoy UberConf because it&#39;s packed full of lots of great sessions, and it&#39;s conveniently local. However, because I go to various local user groups and attend so often, I find that, if I go two years in a row there are too many sessions I&#39;ve seen before, and I wind up disappointed. So for the past few years, I&#39;ve been alternating between UberConf and something new. Two years ago, it was <a href="http://www.nomachetejuggling.com/2013/07/30/oscon-2013-a-review/">OSCON</a>, and this year it was <a href="https://qconnewyork.com/">QCon New York</a>.</p> <p>I chose QCon for a few reasons. One, the sessions seemed very focused on architecture and higher-level concepts, with very few language/technology talks. This was right up my alley because, while there are some languages and tools I&#39;d like to go deeper on, I think a more significant area for improvement for me is architecture and scalability. We get tons of traffic at my job - more than any other place I&#39;ve ever worked - so I&#39;ve had to learn a lot about scalability, and the nature of the work has forced me to really see broad system design differently.</p> <p>I went to QCon specifically wanting to improve some areas where I was weak, namely containerization, microservices, and reactive programming. I hear a lot of buzz about these things, and they pop up on recent <a href="http://www.thoughtworks.com/radar/a-z">ThoughtWorks Technology Radar</a>s, and QCon seemed to have a lot of sessions to offer in these areas. It took a LOT of convincing to get my employer to agree to send me to a conference location as expensive as New York, but eventually they approved it. Here I will detail some of my thoughts about the experience, in case it may be of use to others considering QCon.</p> <!--more--> <h1>Sessions</h1> <p>First and foremost, the sessions. Networking isn&#39;t my thing, I&#39;m all about the quality, quantity, and variety of sessions offered. <strong>I picked QCon based on the sessions, and I was not disappointed.</strong></p> <p>The sessions were well-arranged into tracks, as is common with these kinds of conferences. What was somewhat different about QCon, at least from my perspective, was how cohesive sessions were within a track, and how diverse the tracks themselves were. A lot of times tracks are really general, like &quot;Java&quot; or &quot;Agile&quot;, or they can be too similar to each other. In QCon&#39;s case, all of the tracks themselves were very different, but very specific, like &quot;Fraud Detection and Hack Prevention&quot; and &quot;High Performance Streaming Data&quot;. Within a track, all of the talks were very closely related, and it actually made sense to pick a track and stick with it, rather than buffet style picking-and-choosing based on session alone.</p> <p>The sessions were the perfect length. I&#39;ve complained before that UberConf&#39;s 90 minute sessions can sometimes seem overlong or padded, and that OSCON&#39;s 30-minute sessions seemed rushed or abbreviated right when they were getting good, but QCon strikes a great balance at 50 minutes each. This is short enough to prevent topic fatigue, but long enough to go in depth. Speakers usually did a great job of giving a presentation in-line with the topic title and description as well, which is (somewhat surprisingly) rare for tech conferences.</p> <p>One complaint is that <strong>slides were usually made available AFTER sessions were over</strong>. I hate when this happens, I want to see slides ahead of time, both because I can use them to make sure the content is going to be interesting, and because I use Evernote to actually take notes IN the PDF itself, highlighting or marking up the document with my own notes. The only argument I can imagine for why slides would be held off until after a presentation is that speakers might be modifying the slides until just before they give a talk, but frankly I think that stinks of unpreparedness. Slides should be available in advance, no exceptions.</p> <p>One excellent feature of QCon was that almost all of the talks were published in video form after the sessions were over (usually late at night or the next day). The recording quality was excellent, full video of the speaker and their slides synced up, and actual cameramen that kept the speaker in frame for the whole talk. Audio quality was excellent as well. UberConf does something similar by making audio-only available, but sometimes speakers forget to press record, and I often found myself skipping some sessions with the intent to listen to them on audio later, only to find that they weren&#39;t recorded. QCon solves this problem entirely with a professional A/V staff and quick editing/uploading. What&#39;s more, the slides are available when the video is - <strong>I actually found myself more easily able to take notes on the recorded talks than I was able to when watching talks live.</strong></p> <p>I learned a whole bunch from the sessions I went to, though there was a day (Thursday) where I narrowed down to two talks for every time slot, only to find out based on Twitter that the session I <em>didn&#39;t</em> choose was the better one. This was annoying, but I fixed it with the video recordings.</p> <p>I also really liked there being a special &quot;Modern Computer Science in the Real World&quot; track - it&#39;s rare to see really heavy CS stuff at programmer conferences, I liked it.</p> <table class='image aligncenter' width='646'><tr><td><img src="http://www.nomachetejuggling.com/assets/qcon_sessions.jpg" width='640' height='427'/></td></tr></table> <h2>Mini-Talks</h2> <p>On Wednesday, each of the tracks had a session which was &quot;mini-talks&quot;, kind of a series of lightning talks related to the track. I really, really love lightning talks, I have found that there are a lot of 40-50 minute talks that really should just be 5 or 10 minute lightning talks, and they&#39;re a great way to get wide exposure on a lot of different topics.</p> <p>I wish QCon had done a mini-talks session for every track, rather than just the tracks on the first day of the conference, because <strong>the mini-talks were great</strong>. Again, the video recordings were instrumental here, it was extremely tough to decide which mini-talks to attend, and with the help of video I was able to attend all of them.</p> <h2>Open Spaces</h2> <p>QCon also has an &quot;Open Space&quot; slot for each track, where attendees would get together and brainstorm their own topics, then other attendees would speak about them if they knew anything about the topic.</p> <p>I hate these things, I typically find that they either fizzle out due to not enough participation, or get completely controlled by a single enthusiastic person. I guess a lot of people like Open Spaces, but they just aren&#39;t for me. Every time I saw an Open Spaces talk on the schedule, I wished it was another session, or a session of mini-talks.</p> <h1>Workshops</h1> <p>Like a lot of other conferences, QCon had two days of workshops. Workshops can be really hit or miss in my experience (usually miss), and QCon was no exception. One way QCon did a good job was that Monday was for all-day workshops, and Tuesday was for two half-day workshops (though they had some all-day ones as well). This is a good way to go, personally <strong>I&#39;ve found half-day workshops tend to work better than the all-day variety</strong>.</p> <p>The workshop I picked for Monday wound up being entirely not what was I expecting. It was extremely academic and not very hands-on, which is about the greatest sin a &quot;workshop&quot; can commit. If every participant isn&#39;t on their laptop, it&#39;s not a workshop, it&#39;s just an all-day lecture. QCon was particularly strict about not allowing you to change workshops after making selections, so I was stuck in the workshop the whole day, hating it.</p> <p>The two half-day workshops were much better, but again they suffered from the problem so many workshops have, which is <strong>catering to the slowest, least prepared people in the workshop</strong>. I&#39;ve said it before but if you ignore the e-mail that goes out with the instructions to get set up, you should be left behind. Read the e-mails and do the setup, if you can&#39;t do that then you deserve to have your money wasted; the alternative is that everything slows down so you can catch up meaning everyone else&#39;s money is wasted instead.</p> <p>I especially want to call out <a href="https://twitter.com/everett_toews">Everett Toews</a> for doing an excellent job with his OpenStack half-way workshop. He had a bunch of helpers who got the slower people set up without having to slow down himself, and overall the workshop got a lot accomplished and taught me a lot about OpenStack. I think the session occasionally devolved into &quot;here, copy and paste these commands&quot; lacking explanation, but for the most part the entire workshop was great, and <strong>easily a highlight of the conference for me</strong>.</p> <h1>Keynotes</h1> <p>I tend to expect Keynotes to be extremely non-technical technical talks. Like a talk that you might find scheduled normally, except watered down to a point where every audience member would be okay choosing it. These usually mean I&#39;m not a fan of them.</p> <p>OSCON had an interesting approach to the Keynote, which was a series of short keynotes, almost lightning-talk style, which I really liked. QCon didn&#39;t do this, but they actually managed to avoid the &quot;technical talks with no technical information&quot; by having legitimate technical talks, pretty in-depth from a technical standpoint.</p> <p><strong>Mary Poppendieck&#39;s Microchips to Microservices keynote was fantastic, as was Todd Montgomery and Trisha Gee&#39;s less technical but still highly enjoyable &quot;How Did We End Up Here?&quot; talk</strong>. I disagreed with a lot of points that Todd and Trisha made, but their talk gave me a lot to think about, which is always fun.</p> <p>I wasn&#39;t a big fan of &quot;Driven to Distraction&quot; which was a talk about the different kinds of &quot;X-Driven-Development&quot; there are. This was largely intended to be a comedy talk, rather than a technical talk, and it closed the day rather than opened it. I wasn&#39;t a fan of the humor, honestly. A little into the talk, I realized he was literally going to the entire alphabet, ADD, BDD, CDD, DDD, EDD, and so on. Some letters got multiple definitions, I think I gave up and left around GDD (&quot;Google Driven Development&quot;, because we Google things, har). Kind of ended the conference on a sour note, but something skippable at the end is nice in case your brain is fried, as mine was.</p> <h1>Expo Hall</h1> <p>Like OSCON, QCon had an area for vendors promoting their products and giving stuff away. This room was much smaller than OSCON&#39;s, but it wasn&#39;t off to the side. In fact, you had to walk through to get to 3 of the salons with regular talks, which was somewhat annoying. However, the vendors themselves were a lot less pushy, and only tried to talk to you if you came directly to their book and initiated conversation. They also let you go more easily once you indicated you wanted to move on.</p> <p>I can&#39;t complain much about the vendor room, they gave cool stuff away and getting your nametag scanned for the low low price of being pestered via e-mail later on earned you a spot in a drawing. There were enough giveaways that your chances were fairly decent, in fact I won a Lego Mindstorm set.</p> <table class='image aligncenter' width='646'><tr><td><img src="http://www.nomachetejuggling.com/assets/qcon_haul.jpg" width='640' height='360'/><p class='caption'>Swag!</p></td></tr></table> <p>I still think the coolest version of this I&#39;ve ever seen is at OSCON, where the Expo Hall was gigantic and had all sorts of interesting stuff in it, with a lot of demos and products (even a car). QCon&#39;s room being so much smaller meant that you could go see every booth in just a few minutes, but then were stuck going through the same room multiple times throughout the conference. It also could get extremely crowded in the room, with both booth traffic and just-passing-through traffic, which discouraged me from wanting to spend much time booth hopping.</p> <h1>Attendees</h1> <p><strong>I really liked the class of people at QCon, it seemed like mostly seasoned veterans in the world of software engineering</strong>. There weren&#39;t a lot of young hipsters or brogrammers or anything like that, it seemed like a lot of graybeards. I&#39;m a fan of that, I think our industry generally undervalues people that have been in the field for a while, so it was nice to see such a seasoned group.</p> <p>Of course, I mostly avoided talking to people because I hate networking, so I can&#39;t say much more than that. All I can say is I didn&#39;t overhear conversations that made me cringe, so it seemed like a brighter group of people overall than I&#39;ve seen at some other conferences.</p> <p>One thing I will say, I really liked how QCon set up their lunch tables. Lunch is typically when most of the socializing happens, and OSCON for example has tables for people with similar interests to meet and mingle. QCon had a similar setup, but I appreciated that they also had a handful of rectangular tables pushed against the walls on the perimeter of the lunchroom. A clear set of &quot;I don&#39;t want to mingle&quot; tables. Good stuff if you hate chit-chat like I do.</p> <blockquote class="twitter-tweet" lang="en"><p lang="en" dir="ltr">Just ran into stranger at <a href="https://twitter.com/hashtag/qconnewyork?src=hash">#qconnewyork</a> who recognized my name from my badge because he follows me on Twitter and doesn&#39;t know why. <a href="https://twitter.com/hashtag/surreal?src=hash">#surreal</a></p>&mdash; Rod Hilton (@rodhilton) <a href="https://twitter.com/rodhilton/status/608641289379262464">June 10, 2015</a></blockquote> <script async src="//platform.twitter.com/widgets.js" charset="utf-8"></script> <h1>Venue</h1> <p>Alright, here&#39;s the bad news. The conference was held at the New York Marriott at the Brooklyn Bridge, and <strong>the venue was easily my biggest complaint about the entire conference</strong>.</p> <p>The hotel itself had absolutely no restaurant (or bar for that matter), which made meals a bit difficult. What the Marriott had was a lounge that you could pay to access for $75 per day, which is outrageous. The lounge was the only place to get a meal or a drink in the entire building, and there was no room service offered.</p> <p>Normally only dinners would be a problem in this type of situation, since conferences usually include breakfast and lunch. However, what passed for &#39;Breakfast&#39; was extremely disappointing - 100% pastries and breads, no eggs or meat or protein of any kind (actually they did have bland, ice cold poached eggs). Nothing served warm or even warmable (bagels, no toaster). A bunch of pastries and muffins are not &quot;brain food&quot; - starting a full day of technical talks with a bunch of muffins is a surefire way to be nodding off by the third talk. I wound up walking to a nearby Panera every single morning to get an egg sandwich.</p> <p>Lunch was better, but also not great. I think by choice of QCon itself, the hotel catering had to make everything gluten-free. I find the whole gluten-free for non-celiacs fad <a href="http://www.georgeinstitute.org.au/media-releases/dont-believe-the-hype-on-gluten-free-food">generally irritating</a> but what really made this annoying was what foods they chose to serve. If I say to you &quot;okay we need a vegetarian meal&quot; you try to think of foods that don&#39;t involve meat, you don&#39;t immediately think of a meat dish and substitute tofu in. Similarly, why does your &#39;gluten free&#39; menu consist almost entirely of gluten-free pasta? I wound up grabbing lunch elsewhere for 3 of the 5 days I was at the conference. This was frustrating, because I convinced my employer to send me to NY by arguing that meals were mostly included, but then I wound up having to pay for most meals.</p> <p>Aside from workshops, it was very rare to find tables in the sessions - usually it was just rows of seats. And the walls were those sliding accordion walls that hotels use to divide huge rooms into sections. The end result being, it was very hard to find a place to plug in a laptop, and situate it in a way to take notes. This isn&#39;t really the venue&#39;s fault, I&#39;m sure QCon told them no tables, but it&#39;s always something that irks me as a notetaker. Like I mentioned earlier, I actually found it easier to take notes and watch sessions from the video recordings, from the comfort of my hotel room after hours.</p> <h1>Misc</h1> <p>In summary, QCon absolutely excelled in terms of session quality, variety, and depth, but workshops were an area for improvement (as with every other conference that offers them), and the venue itself was dreadful.</p> <p>I learned a lot at the conference, and was able to gain a lot of insight in the areas where I was hoping to. One particular interesting note was, I went to a lot of &quot;microservices&quot; talks, but almost all of them were &quot;here are some tips for your microservices&quot; and not &quot;here are pros and cons of microservices&quot; or &quot;why you should use microservices&quot;. In other words, most of the microservices talks assumed that you were already building and deploying microservices. This was somewhat shocking to me and made me feel behind the curve a bit - I&#39;m really not sold on microservices, I think the operational concerns are likely to outweigh the benefits, so it was interesting to see it presented like that ship has already sailed. There was a talk by StackOverflow in favor of monoliths, but everything else was all microservices all the time. Martin Fowler recently published a great <a href="http://martinfowler.com/articles/microservice-trade-offs.html">pros-and-cons of Microservices article</a> that I found myself nodding along to.</p> <p>QCon had a nice &#39;custom schedule&#39; builder like OSCON but QCon&#39;s was a bit behind, it didn&#39;t integrate with an app and give notifications like OSCON&#39;s did. However, the full video recordings of sessions available same day were phenomenal, and the ability to share with 10 other people who didn&#39;t attend the conference is awesome.</p> <p><strong>Overall, I really enjoyed my QCon experience and it&#39;s definitely on my radar for a conference to attend again in the future</strong>. However, most of the things I disliked about the conference were related to its physical location and venue, which makes me wonder if it&#39;s worth the (very high) price tag since the best part (the recorded sessions) are eventually made available on the internet.</p> Wed, 01 Jul 2015 00:00:00 -0400 http://www.nomachetejuggling.com/2015/07/01/qconny-2015-a-review/ http://www.nomachetejuggling.com/2015/07/01/qconny-2015-a-review/ Uploading a Jekyll Site to Rackspace Cloudfiles <p>This blog was never intended to be popular by any stretch of the imagination. Largely I started it simply to have a place to gather solutions to technical problems I&#39;ve encountered, so that I could easily look those solutions up if I needed them again. The blog has always run on my own shared hosting server, on a self-installed version of Wordpress.</p> <p>To my great surprise, a <a href="http://www.reddit.com/r/programming/comments/2986e4/the_worst_programming_interview_question/">few</a> <a href="http://www.reddit.com/r/programming/comments/hn1fx/a_different_kind_of_technical_interview/">of</a> <a href="http://www.reddit.com/r/programming/comments/yvr9/my_interview_with_google/">my</a> <a href="http://www.reddit.com/r/TrueReddit/comments/q98ld/the_star_wars_saga_suggested_viewing_order_iv_v/">posts</a> have found their way to the front page of <a href="http://www.reddit.com/domain/nomachetejuggling.com">reddit</a>. My <a href="http://www.nomachetejuggling.com/2011/11/11/the-star-wars-saga-suggested-viewing-order/">post about Star Wars</a> has been mentioned on <a href="https://www.youtube.com/watch?v=effD1u4oCRE">King of the Nerds</a> and <a href="https://www.youtube.com/watch?v=keSFjjhUyVA">The Big Bang Theory</a>, and even landed me an <a href="http://www.npr.org/2014/03/20/291977042/theres-more-than-one-way-to-watch-star-wars">Interview on NPR</a>. </p> <table class='image aligncenter'><tr><td><script type="text/javascript" src="//www.google.com/trends/embed.js?hl=en-US&q=Machete+Order&cmpt=q&content=1&cid=TIMESERIES_GRAPH_0&export=5&w=600&h=330"></script></td></tr></table> <p>Needless to say, the traffic to my blog has been both extremely unexpected and unpredictable. The Star Wars post had been online for months with virtually no traffic before <a href="http://archive.wired.com/geekdad/2012/02/machete-order-star-wars/">Wired</a> suddenly linked to it, instantly decimating my web server. I&#39;ve fought and fought with various configurations for Wordpress, used <a href="http://wordpress.org/plugins/w3-total-cache/">as much caching</a> as possible, and even had my <a href="https://www.servint.net/">web host</a> temporarily upgrade my service, all trying to keep a web site that makes no money online even when traffic increases by a factor of 100 overnight. <strong>When my site goes down, it&#39;s embarrassing, because even though it&#39;s just a personal blog on a shared host, it gives the impression that I, as a software developer, don&#39;t know how to make a web site scale</strong>.</p> <h1>Switching to Jekyll</h1> <table class='image alignright' width='306'><tr><td><img src="http://www.nomachetejuggling.com/assets/jekyll-logo.png" width='300' height='141'/></td></tr></table> <p>So after the most recent pummeling I took due to a <a href="https://news.ycombinator.com/item?id=7953725">Hacker News link</a>, I decided it was time to <strong>bite the bullet and convert the entire site to <a href="http://jekyllrb.com/">Jekyll</a></strong>. I&#39;ve messed around with the technology before to build another, smaller, blog, so I was somewhat familiar with the constructs and idioms. A lot of work and ten custom plugins later, the entire site was converted, with very little loss of functionality.</p> <!--more--> <p>I didn&#39;t want to serve the files from my shared host because I know firsthand that the traffic spikes I experience are often enough to overwhelm apache itself, and I couldn&#39;t host it with <a href="https://pages.github.com/">GitHub Pages</a> due to the aforementioned ten custom plugins. I&#39;ve used both Amazon S3 (to host the smaller Jekyll-based blog) and Rackspace Cloudfiles (as a CDN for the Wordpress version). Of those two, I find Amazon S3 to be extremely overcomplicated and difficult to work with, but there&#39;s a great <a href="https://github.com/laurilehmijoki/s3_website">S3_Website</a> gem that makes uploading a Jekyll blog a snap. Rackspace Cloudfiles is much more straightforward to work with, but the <a href="https://github.com/nicholaskuechler/jekyll-rackspace-cloudfiles-clean-urls/blob/master/cloudfiles_jekyll_upload.py">Python script</a> that <a href="http://www.rackspace.com/blog/running-jekyll-on-rackspace-cloud-files/">even Rackspace itself</a> links to has given me various dependency issues.</p> <p><span data-pullquote="Rackspace Cloudfiles is a bit cheaper per GB than Amazon S3, and ... that became the deciding factor. " class="left">Rackspace Cloudfiles is a bit cheaper per GB than Amazon S3, and ... that became the deciding factor. </span> In the end, Rackspace Cloudfiles is a bit cheaper per GB than Amazon S3, and since this blog receives a nontrivial amount of traffic, that became the deciding factor. Since I always had issues with the python script that uploads a Jekyll blog to Cloudfiles, I decided to do some research into alternative means of automated uploading (<a href="http://cyberduck.io/">Cyberduck</a> works, but I wanted something that I could make Jenkins run).</p> <p>Unfortunately, <strong>almost everything I found wound up linking to the exact same Python script that gave me trouble</strong>. So I decided to write my own, which I&#39;m open-sourcing for the benefit of anyone else that has had similar problems.</p> <h1>jekyll-cloudfiles-upload</h1> <p><a href="https://github.com/rodhilton/jekyll-cloudfiles-upload">jekyll-cloudfiles-upload is hosted on GitHub</a> and is a single Ruby script that can be dropped into your Jekyll blog project directory. It will look in <code>_site</code> for all of your static site files, compare them to what is in your Rackspace Cloudfiles container, upload any that need updating, and delete anything in the container you no longer have. It only has a few small dependencies (ruby and a ruby gem named <code>Fog</code>), and I&#39;ve been using it to update this blog with great success.</p> <h2>Installation and Usage</h2> <ol> <li><p>Log into Rackspace Cloud Files and create your container. <em>You must create your container first, the script will not do that</em>.</p> <blockquote> <p><strong>Pro-Tip</strong>: Before you upload anything, set your container&#39;s TTL to something other than the default, which is 72 hours. Once a file is loaded into the CDN, it seemed to me that, even if you changed your container&#39;s TTL after the fact, the TTL change itself wouldn&#39;t propagate until after 72 hours. Changing it first (I use 15 minutes) before uploading files seemed to fix this issue.</p> </blockquote></li> <li><p>Install <code>fog</code> rubygem via <code>gem install fog</code></p></li> <li><p>Put a <code>.fog</code> file in your home directory that looks like this (it&#39;s a yaml file, be careful not to use tabs instead of spaces):</p> <div class="highlight"><pre><code class="language-yaml" data-lang="yaml">default: rackspace_username: your_user_name rackspace_api_key: your_api_key rackspace_region: your_preferred_region </code></pre></div> <p>The rackspace regions are strings like &#39;iad&#39; or &#39;dfw&#39;, depending on your preferred container region. You can get your api key from the Rackspace control panel&#39;s Account page.</p> <p>If you have multiple sites with multiple containers all in different regions, you&#39;ll have to hand-alter the script so that it doesn&#39;t look up this information in Fog, but just hardcodes it instead. If you do this, I suggest using the ruby symbol syntax in the <code>cloudfiles_upload.rb</code> script, such as <code>:iad</code>.</p></li> <li><p>Copy the <code>cloudfiles_upload.rb</code> script from the GitHub repository into the directory for your Jekyll project. It&#39;s a good idea to also make it executable via <code>chmod a+x cloudfiles_upload.rb</code></p></li> <li><p>Build your site via <code>jekyll build</code></p></li> <li><p>Execute <code>./cloudfiles_upload.rb container_name</code> or <code>ruby cloudfiles_upload.rb container_name</code>.</p> <p>The script will spider through the <code>_site</code> subdirectory and look for any files that need to be added, deleted, or updated. Only files whose md5 hashes differ will from those in the container will be uploaded, so it will not upload files unnecessarily.</p> <p><strong>Note</strong>: You may optionally leave off the <code>container_name</code> parameter, and the script will use the name of the directory you are in. So if you name your directory and container <code>mysite.com</code>, you can just run <code>./cloudfiles_upload.rb</code> with no arguments.</p> <blockquote> <p><strong>Pro-Tip</strong>: Add <code>cloudfiles_upload.rb</code> to your <code>_config.yml</code> file&#39;s exclusion list so it doesn&#39;t get uploaded.</p> </blockquote></li> </ol> <h1>Dogfooding</h1> <p>I offer no guarantee of support on this script, but I can assure you that I&#39;m dogfooding the hell out of it. I set up a private Jenkins instance that watches for changes to my private <a href="https://bitbucket.org/">BitBucket</a> repository that contains this blog. The repository has <code>jekyll-cloudfiles-upload</code> as a submodule, with the <code>cloudfiles_upload.rb</code> script symlinked to the submodule&#39;s version. Any change to the blog pulls down the most recent copy of the script, builds the blog, and then runs the script to upload it.</p> <p>I liked this solution so much that I wound up converting the smaller blog that I had been running on Amazon S3 over to Rackspace Cloudfiles as well. I also have a Jenkins job that looks for changes to the <code>jekyll-cloudfiles-upload</code> project, and automatically kicks off the jobs for both web sites whenever it changes as well, so this script is definitely instrumental to a process that controls a web site whose downtime personally embarrasses me a great deal. Again, no guarantees, but I&#39;m putting a lot of trust in this script, for whatever that&#39;s worth.</p> <h1>Jekyll Thoughts</h1> <p>So far, I&#39;m digging Jekyll a lot. I&#39;d used it before for the smaller blog as I mentioned, but that was my first, so I used <a href="http://jekyllbootstrap.com/">JekyllBootstrap</a> thoroughly. It was good for getting set up, but I found making modifications to themes and general customization quite perplexing and difficult. This time, I built everything from scratch, including all of the custom plugins I&#39;m using, and I have a much better understanding of how Jekyll works.</p> <p>The only thing I had to give up was the rightmost sidebar. Previously, that area actually showed my latest tweet, and various updates from my Goodreads, Trackt, Last.fm, Groovee, and Diigo feeds. Those used the <a href="https://wordpress.org/plugins/better-rss-widget/">Better RSS Widget</a> Wordpress plugin, and I liked the feature but it would occasionally have trouble pulling feeds, causing it to leave an error on the cached version of a page for hours until the cache cleared. I&#39;m alright with my sidebar-o-social-icons that I replaced it with, though.</p> <p><span data-pullquote="I love writing posts in Markdown. " class="right">I love writing posts in Markdown. </span> I&#39;ve always wanted to be able to do that with Wordpress, but found that plugins which supported it were generally terrible. I wish it was easier to make custom alterations to the markdown processing, but <a href="http://jekyllrb.com/docs/plugins/#tags">Jekyll Tags</a> are a decent workaround. Like I mentioned earlier, I&#39;ve got a lot of custom plugins gluing this site together, but I&#39;m happy with the readability of my markdown source files, and I like that there&#39;s an abstraction layer translating those to HTML rather than embedding HTML directly into posts or writing Wordpress shortcode processors.</p> <p>I may eventually put some of these plugins into GitHub as well, and I wound up writing a pretty handy extension to <a href="http://highlightjs.org/">highlight.js</a> that makes it easier to copy and paste syntax highlighted code which I think others might find useful. But easily, the most useful thing I wrote to support this effort - aside from a highly customized script that ran my blog post&#39;s html files through forty regular expressions to convert them to markdown - was the <code>cloudfiles_upload.rb</code> script. Hopefully others may find it useful as well.</p> Fri, 04 Jul 2014 00:00:00 -0400 http://www.nomachetejuggling.com/2014/07/04/uploading-a-jekyll-site-to-rackspace-cloudfiles/ http://www.nomachetejuggling.com/2014/07/04/uploading-a-jekyll-site-to-rackspace-cloudfiles/ Top 10 Career-Changing Programming Books <p>When I graduated with a Computer Science degree ten years ago, I was excited to dive into the world of professional programming. I had done well in school, and I thought I was completely ready to be employed doing my dream job: writing code. What I discovered in my very first interview, however, was that I was massively underprepared to be an actual professional programmer. I knew all about data structures and algorithms, but nothing about how actual professional, &quot;enterprise&quot; software was written. I was lucky to find a job at a place willing to take a chance on me, and proceeded to learn as much as I could as quickly as I could to make up for my deficiencies. This involved reading a LOT of books.</p> <p>Here I reflect on my 10-year experience programming professionally and all of the books I&#39;ve read in that time, and offer up the ten that had the most profound impact on my career. Note that these are not the &quot;10 best&quot; programming books. I do feel all of these books are very good, but that&#39;s not the only reason I&#39;m selecting them here; I&#39;m mentioning them because I felt that I was a profoundly different person after reading each than I was beforehand. Each of these books forced me to think differently about my profession, and I believe they helped mold me into the programmer I am today.</p> <p>None of these books are language books. I may feel like learning to program in, say, Scala, had a profound impact on how I work professionally, but the enlightening thing was Scala itself, not the book I used to help me learn it. Similarly, I&#39;d say that learning to use Git had a significant impact on how I view version control, but it was Git that had the impact on me, not the book that I used to teach myself the tool. The books on this list are about the the content they dumped into my brain, not just a particular technology they taught me, even if a technology had a profound impact on me.</p> <p>So, without further ado...</p> <h1>Top &quot;10&quot;</h1> <h2><a href="http://www.amazon.com/The-Pragmatic-Programmer-Journeyman-Master/dp/020161622X">The Pragmatic Programmer</a></h2> <table class='image alignright' width='244'><tr><td><a href='http://www.amazon.com/The-Pragmatic-Programmer-Journeyman-Master/dp/020161622X'><img src="http://www.nomachetejuggling.com/assets/pragprog-238x300.jpg" width='238' height='300'/></a></td></tr></table> <p>I know, I know. Every list you&#39;ve ever seen on the internet includes this book. I&#39;m sorry, I wish I could be more original, but this book really is an eye-opener. <em>The Pragmatic Programmer</em> contains 46 tips for software professionals that are simply indispensable. As the name implies, the book avoids falling into any kind of religious wars with its tips, it&#39;s simply about pragmatism.</p> <p>If you were to read only one book on this list, this is the one to read. It never goes terribly deep into anything, but it has a great breadth, covering the basics that will take a recent college-grad and transform him or her into someone employable, who can be a useful member of a team.</p> <p>Many programmers got into the field because they liked hacking on code in their spare time, writing scripts to automate tasks or otherwise save time. There is a set of skills one develops just to sling code that makes a computer perform specific tasks, and that exact same skillset is needed by many, many employers. But there are many people who see programming professionally as simply an extension of their hobby, and do things the same way whether they are hacking at home or at work. <em>The Pragmatic Programmer</em> permanently altered how I view programming, it&#39;s not just extending my hobby of coding and getting people to pay me for it; there&#39;s a fundamental line between professional coding and hobbyist coding, and I am able to see that line and operate differently depending on what side of it I&#39;m on thanks to <em>The Pragmatic Programmer</em>.</p> <p>How groundbreaking is this book? Groundbreaking enough that it launched an entire publishing company. It&#39;s a big deal, if you&#39;ve somehow managed not to read it yet, go do so.</p> <p><strong>What it changed:</strong> How I view &quot;programming&quot; as a job instead of a hobby I get paid for.</p> <h2><a href="http://www.amazon.com/Continuous-Delivery-Deployment-Automation-Addison-Wesley/dp/0321601912">Continuous Delivery</a></h2> <table class='image alignright' width='233'><tr><td><a href='http://www.amazon.com/Continuous-Delivery-Deployment-Automation-Addison-Wesley/dp/0321601912'><img src="http://www.nomachetejuggling.com/assets/continuousDelivery-227x300.jpg" width='227' height='300'/></a></td></tr></table> <p>Releasing software is one of the most stressful parts of the job. I can&#39;t tell you how many times in my career I&#39;ve been part of a botched launch, or up until the wee hours of the morning on a conference call trying to get software into the hands of customers. When do we branch, what goes in what branch, how do we build the artifacts, what process do we walk through to get them where they need to go? It can be one of the most complex, error-prone, and difficult parts of professional programming.</p> <p><em>Continuous Delivery</em> means to do away with all of that difficulty. It describes a mindset, toolset, and methodology for completely turning releases on their head. Instead of doing them less frequently because they are difficult, do them more frequently so they&#39;re forced to be easier. In fact, don&#39;t just do them more frequently, do them <strong>all the time</strong>. <em>Continuous Delivery</em> describes, with real-world practical examples, how to version control all configuration, how to test integration points, how to handle branching and branch content, how to safely rollback, how to deploy with no downtime, how to do continuous testing, and how to automate everything from checkin to release.</p> <p>In a lot of ways the book describes a pie-in-the-sky ideal. It&#39;s difficult to achieve truly continuous delivery, though GitHub, Flickr, and many other companies seem to have done so. But as the old adage goes, aim for the moon, even if you miss you&#39;ll end up among the stars. Wait, that adage is insane, stars are further away than the moon. Who came up with that phrase? Where was I? Oh right, even if you don&#39;t ever reach the true ideal, every step you made toward it makes deployments at your company that much better. I&#39;ve worked in various environments where the principles of this book have been applied at different levels, and I can personally attest that there is a near-perfect linear relationship between how much you adhere to the advice in this book, and how smoothly releases go.</p> <p>I worked in an environment operating at about a 70%-level of adherence to the philosophy outlined in this book, and it was heaven. When I left that job, my new employer was at approximately 0%, and it was complete misery. I set about implementing the ideas of the book and even a 10%-level of adherence was like a fifty-ton boulder being removed from my back. It worked so well that it was like a blinding light of epiphany for co-workers, and we wound up hiring someone whose sole job it was to help get us further along. Today we&#39;re at about 50%, and it&#39;s easily five times better than it was at 10%, and infinitely better than at 0%. Still hoping to get to 100%, obviously, but there&#39;s no doubt that every aspect of the book makes releases smoother and less stressful. I simply don&#39;t think I could ever work any other way ever again, it&#39;s like finding out you&#39;ve been coding with a blindfold on for years.</p> <p><strong>What it changed:</strong> How I release software and bake releasability into my code.</p> <h2><a href="http://www.amazon.com/Clean-Code-Handbook-Software-Craftsmanship/dp/0132350882">Clean Code</a> / <a href="http://www.amazon.com/The-Clean-Coder-Professional-Programmers/dp/0137081073">The Clean Coder</a></h2> <table class='image alignright' width='306'><tr><td><img src="http://www.nomachetejuggling.com/assets/cleanduology-300x191.jpg" width='300' height='191'/></td></tr></table> <p>Look at this, only a few items into my list and I&#39;ve already cheated by including two books as a single entry. Yes, <em>Clean Code</em> and <em>The Clean Coder</em> are two separate books, but honestly they&#39;re both very short, and very similar. Both books are about how a programmer should conduct him or herself professionally, they simply cover different aspects. Professional software developers communicate with their coworkers in two ways: through code and through everything else. <em>Clean Code</em> is about how you communicate with your co-workers (fellow programmers) through code itself, and <em>The Clean Coder</em> is about how you communicate verbally, or through e-mail.</p> <p>Both of these books, written by &quot;Uncle&quot; Bob Martin, really could easily be a single book with two large sections. Bob&#39;s philosophy toward professional software development is honest and direct, some would even say blunt. He makes no bones about it: fail to communicate in the way he describes, and you&#39;re bordering on professional negligence. It seems harsh but, frankly, I&#39;m convinced. Call me a believer. </p> <p>I definitely treat my code differently in light of his suggestions from <em>Clean Code</em>. It may seem strange that I categorize <em>Clean Code</em> as a book about communication, given that it&#39;s all about how to write code. But in the words of Abelson and Sussman, &quot;Programs should be written for people to read, and only incidentally for machines to execute.&quot; Machines will run code whether it&#39;s &quot;clean&quot; or not, but your coworkers will only be able to understand and work with your code if it&#39;s clean. <em>Clean Code</em> is about how to structure your code for others to read, or even for the future version of yourself to read.</p> <p>Even more than <em>Clean Code</em>, <em>The Clean Coder</em> had a profound impact on me. It drastically altered how I talk to bosses, product owners, project managers, marketers, salespeople, and other non-programmers. It advocates taking ownership of your screwups, being honest about abilities and deadlines, and up-front about costs. Not every co-worker you encounter will appreciate the approach outlined in <em>The Clean Coder</em>, but ultimately your customers will, because your products will be better for it.</p> <p><strong>What they changed:</strong> How I conduct myself professionally.</p> <h2><a href="http://www.amazon.com/Release-It-Production-Ready-Pragmatic-Programmers/dp/0978739213">Release It!</a></h2> <table class='image alignright' width='256'><tr><td><a href='http://www.amazon.com/Release-It-Production-Ready-Pragmatic-Programmers/dp/0978739213'><img src="http://www.nomachetejuggling.com/assets/releaseit-250x300.jpg" width='250' height='300'/></a></td></tr></table> <p>A product&#39;s life doesn&#39;t begin when you first create the source code repository, or write the first line of code, or even finish the first story. It begins as soon as it&#39;s launched into production, into the hands of real users. Everything before that is just bits, just plain text files on disks. So in a lot of ways, it&#39;s astonishing how much thought is put into the code for the period of time before it&#39;s really born.</p> <p><em>Release It!</em> places the stress on the real life of a program. It&#39;s all about monitoring, health checking, logging, and ensuring that applications remain operational. It&#39;s about baking in concern for capacity and stability from the start, and what needs to be done to keep a program operating even when there are outages, or broken integrations, or massive spikes in load. Most of all, it&#39;s about <strong>assuming</strong> that code will fail, backend servers will die, databases will timeout, and everything your software depends on will eventually go to hell. It&#39;s a completely different approach to software development, and it&#39;s completely eye-opening.</p> <p>Not to be too pejorative, but if you do enterprise application development, you probably shouldn&#39;t write another line of code before you read this book. I consider pretty much everything I&#39;ve written before it to be inadequate for real production use, even all the stuff currently in production. It covers patterns and anti-patterns to support (or subvert) stability as well as capacity, and the section of the book covering these topics is simply excellent. But then it goes beyond that to also discuss Operational enablement. Even if you&#39;re not into DevOps, and don&#39;t want to really be involved in DevOps work, this book gives you the tools and tips to do what aspect of DevOps is the purview of pure developers. </p> <p><em>Release It!</em>&#39;s tactics will make you your operations team&#39;s favorite person, and greatly help cover you and your teams ass in the eventual case of catastrophic failure somewhere. The patterns sections alone are worth the price of admission here, and the fact that the book is chock full of even more useful content beyond them is kind of stunning. </p> <p><strong>What it changed:</strong> What I consider to be &quot;production-ready&quot;, and how I view Operations.</p> <h2><a href="http://www.amazon.com/First-Design-Patterns-Elisabeth-Freeman/dp/0596007124">Head First Design Patterns</a> / <a href="http://www.amazon.com/Patterns-Enterprise-Application-Architecture-Martin/dp/0321127420">Patterns of Enterprise Application Architecture</a></h2> <table class='image alignright' width='306'><tr><td><img src="http://www.nomachetejuggling.com/assets/patterns-300x189.jpg" width='300' height='189'/></td></tr></table> <p>No list like this would be complete without a book about design patterns. But where&#39;s the famous &quot;Gang of Four&quot; book, you ask? Not on this list, that&#39;s where. Honestly, GoF was a pretty groundbreaking book at the time, but I personally think the presentation of the information it contains is awful. I believe everything presented in GoF is presented better in <em>Head First Design Patterns</em>. I know that not everyone is crazy about the Head First series, and even I find the structure and layout of the book grating at times, but I think the diagrams and visuals are light years better than those of GoF. </p> <p>I also think Head First does a better job of providing <em>contextual</em> examples. While GoF provides sample code implementing the pattern, I feel that <em>Head First Design Patterns</em> provides a more valuable context for its examples, with more explanation about what the code is doing and what it&#39;s for. This helps readers understand <em>when</em> to use specific patterns, which I feel is the most important thing to learn when learning patterns. Too often, people read their first design patterns book and immediately decide to implement as many as they can. This is the wrong approach to take with patterns, and I think Head First&#39;s contextualization and strong visuals make it easier for readers to avoid this mistake. <a href="http://www.codinghorror.com/blog/2005/09/head-first-design-patterns.html">Jeff Atwood disagrees</a> and I can see his point, but I think overall this book is better in this regard than the classic GoF.</p> <p><em>Patterns of Enterprise Application Architecture</em> is the GoF book, but at the level of architecture rather than code. Like GoF, it is extremely dry, and somewhat difficult to get through cover to cover, working better as a reference book than a reading book. It does a very good job, however, of managing to still provide ample context, describing when you&#39;d want to use (or avoid) a particular pattern. I can&#39;t tell you how many times I&#39;ve referenced this book.</p> <p>Patterns provide great &quot;templates&quot; to use when solving common problems. They need to reached for with great care to avoid overuse, but when utilized appropriately can give developers a great deal of confidence in the time-tested designs they outline. Additionally, they provide a shared vocabulary among developers that greatly aids communication about complex topics. Describing the exact kind of hamburger you want to a Burger King employee is difficult when you have to describe every single element of the meal, but it&#39;s much easier when you can simply say &quot;number 5&quot; and you both know exactly what is being ordered.</p> <p><strong>What they changed:</strong> How I design and discuss my software, both at the code and architecture level.</p> <h2><a href="http://www.amazon.com/Working-Effectively-Legacy-Michael-Feathers/dp/0131177052">Working Effectively with Legacy Code</a></h2> <table class='image alignright' width='232'><tr><td><a href='http://www.amazon.com/Working-Effectively-Legacy-Michael-Feathers/dp/0131177052'><img src="http://www.nomachetejuggling.com/assets/legacycode-226x300.jpg" width='226' height='300'/></a></td></tr></table> <p>My first job out of college was replacing a developer who had left the company, as the sole responsible engineer on a massive and extremely complex codebase. Working in this codebase was terrifying, any change I made had the potential to break almost anything, and there was no way to test any changes without pushing a jar to the production system and watching it go. I checked over every change I made about a thousand times, and hand-constructed little <code>public static void main</code> classes just to instantiate classes and invoke methods, and then hand-check results. I had never heard of unit tests at this point (evidently, neither had my predecessor), so everything was done with kid gloves.</p> <p>It wasn&#39;t until 2 jobs later that I actually read <em>Working Effectively with Legacy Code</em>, which describes exactly how to deal with systems like these. The book explains how to take yourself from having no confidence in the codebase or your changes, to having complete confidence in them. It&#39;s not simply about how to effectively manage yourself in the hole you&#39;ve found yourself in, but exactly the tactics you can use to dig yourself out of the hole. It&#39;s organized extremely well, indexed largely by actual complaints you might have about an inherited codebase. If I&#39;d read this book earlier, my first job experience would have been much less stressful, and much more rewarding.</p> <p>One important thing to realize is that &quot;Legacy Code&quot; doesn&#39;t refer exclusively to million-line Cobol codebases. As soon as code is written and deployed somewhere, it&#39;s legacy code from that point forward. Every codebase you&#39;ve worked on that you didn&#39;t write yourself as a greenfield project is a legacy codebase, and the methodology of the book will help. Once upon a time in my career, inheriting another developer&#39;s codebase was frightening for me, and I&#39;d often react (as so many developers do) by immediately wanting to do a full-scale rewrite of any codebase that&#39;s too complex for me to manage. Thanks to this book, I have no problem inheriting code written by others, even if they&#39;re no longer around.</p> <p>Moving to a new job is less intimidating to me now, and I often spend the first few months of my time somewhere new simply getting the scaffolding in place to make changes confidently later on, increasing unit test coverage and breaking code into smaller and more isolated chunks. The full-scale rewrite is no longer the first tool I reach for in my toolbelt, it&#39;s the last one, and I feel confident that I can refactor nearly any codebase into something I&#39;m comfortable working on.</p> <p><strong>What it changed:</strong> How I feel about inherited codebases, and how I manage my confidence working with them.</p> <h2><a href="http://www.amazon.com/Refactoring-Improving-Design-Existing-Code/dp/0201485672">Refactoring</a> / <a href="http://www.amazon.com/xUnit-Test-Patterns-Refactoring-Code/dp/0131495054/">xUnit Test Patterns</a></h2> <table class='image alignright' width='306'><tr><td><img src="http://www.nomachetejuggling.com/assets/refactoring_xunit-300x196.jpg" width='300' height='196'/></td></tr></table> <p>I think most recent college graduates, myself included at the time, are &quot;cowboy&quot; coders. I used to have all the changes in my head, and just tried to get them fed from my brain into the compiler as quickly as I could, before I forgot all the stuff I wanted to do. Today, I cringe when I think about how many characters of code I&#39;d type between actually running or testing my software; &quot;waiting for the compiler is just going to slow me down, let me get all the code written first and then I&#39;ll debug it!&quot;</p> <p>Learning the technique of refactoring, in which you change the structure of code without changing the behavior, forces a mental split. You realize that &quot;coding&quot; is really two jobs, and that structure and behavior should be altered and tested independently, never at the same time. Martin Fowler&#39;s <em>Refactoring</em> is a collection of structure-but-not-behavior changes that really provides the toolset for a lot of other books on this list. <em>Refactoring</em> is so important that, depending on what language you work with, you may not even think you have to actually read it: your IDE probably supports many of the operations it describes out of the box. Nonetheless, it is a critical read, as it puts the reader in the mindset to understand the two hats they must wear as a coder, and how to intentionally change from &quot;coding&quot; to &quot;refactoring&quot;.</p> <p>Of course, refactoring goes hand-in-hand with unit testing. There are hundreds of books covering unit tests and test-driven development, but none of them that I&#39;ve seen break things down as well as <em>xUnit Test Patterns</em>. The book covers everything a programmer needs to become a unit testing badass, how to work with mocks and stubs, how to recognize problem smells in tests, how to refactor tests, and tons more. It&#39;s not about a specific technology or tool, it&#39;s about unit testing best practices in general, and my attitude toward testing and the kinds of tests I write are much improved because of it.</p> <p>Refactoring and testing are essential tools in the programmer&#39;s toolchest, and these two books cover all of the mechanics and tools one needs to master those tools. <em>Refactoring</em> focuses on improving the structure of your code, <em>xUnit Test Patterns</em> focuses on improving the structure of your tests, and your code and tests form a symbiotic bond of code quality. These two books are, in a lot of ways, two sides of a very important coin.</p> <p><strong>What they changed:</strong> How I approach altering existing code, and how I ensure I&#39;ve done so correctly.</p> <h2><a href="http://www.amazon.com/The-Passionate-Programmer-Remarkable-Development/dp/1934356344">The Passionate Programmer</a> / <a href="http://www.amazon.com/Land-Tech-Love-Pragmatic-Life/dp/1934356263">Land the Tech Job You Love</a></h2> <table class='image alignright' width='306'><tr><td><img src="http://www.nomachetejuggling.com/assets/career_books-300x225.jpg" width='300' height='225'/></td></tr></table> <p>Okay, I get it, I&#39;m terrible at making these lists, and clearly should have just done a &quot;Top 15&quot; or something. In any case, landing that first job out of college is tough, but eventually the day comes when it&#39;s time to move on. <em>The Passionate Programmer</em> is largely about how to find the right kind of job for you, what to look for in tech companies, and how to manage the direction of your career. It&#39;s pretty high level, but full of extraordinarily important advice to ensure you find yourself at companies that fit you and that you fit into well. <em>Land the Tech Job You Love</em> is more about the mechanics of this process, how to write a resume, how to interview, how to negotiate a salary, and the like. This is another situation where really two books are so closely related that they&#39;d be better as a single larger book.</p> <p>These books helped give me confidence to understand the process of hunting for and getting a job as a programmer. It completely shifted my mentality, from being the unqualified person begging a company to give me a job, to being a competent and capable engineer simply searching for a mutually beneficial fit. It changed how I view the job hunt, and how I conduct myself in interviews. After reading these books, I completely scrapped my entire resume and created a new one from scratch.</p> <p>In a lot of ways, these books inspired me to create this very blog, or at least adjust what I used it for. I view my various online profiles as part of my &quot;brand&quot; and I think my viewpoint shift in this regard informs a great deal of what I post here, on twitter, and elsewhere. Yes, even all the inappropriate swearing (companies should probably know what they&#39;re getting into with me). </p> <p>I have a lot of confidence about my career now, and I don&#39;t live in fear of losing my job or being unable to find a new one. I think about my career differently, as a very planned and deliberate thing, not just a series of jobs. It makes me excited about my future as a programmer, rather than concerned and fearful, which is a liberating sensation.</p> <p><strong>What they changed:</strong> How I view and manage my career.</p> <h2><a href="http://www.amazon.com/Apprenticeship-Patterns-Guidance-Aspiring-Craftsman/dp/0596518382/">Apprenticeship Patterns</a></h2> <table class='image alignright' width='234'><tr><td><a href='http://www.amazon.com/Apprenticeship-Patterns-Guidance-Aspiring-Craftsman/dp/0596518382/'><img src="http://www.nomachetejuggling.com/assets/apprenticeship-patterns-228x300.jpg" width='228' height='300'/></a></td></tr></table> <p><em>Apprenticeship Patterns</em> isn&#39;t really a patterns book as the name implies, but it&#39;s content has been kind of shoehorned into the format, I assume to increase sales. Ignoring that flaw, <em>Apprenticeship Patterns</em> is the best book on Software Craftsmanship I&#39;ve read, and I&#39;ve read quite a few. I actually recommend it above <a href="http://www.amazon.com/Software-Craftsmanship-The-New-Imperative/dp/0201733862">Pete McBreen&#39;s Software Craftsmanship</a>, because it covers pretty much everything useful from that book, but excises some of the more unrealistic or naive bits, as well as the extremely long and pointless section about salary. <em>Apprenticeship Patterns</em> is a bugfix release for <em>Software Craftsmanship</em>.</p> <p>This book was the one that made really see the value in the Software Craftsmanship movement, and truly embrace it. I&#39;ve written elsewhere about <a href="http://www.nomachetejuggling.com/2013/02/02/brief-thoughts-on-software-craftsmanship/">why I like the Software Craftsman title</a>, but this was the book that convinced me to consider myself part of that crowd. Software Craftsmanship isn&#39;t just about what customers can expect from you, it&#39;s about what your fellow developers can expect from you, and what you should expect from yourself. It&#39;s not just about writing clean code, it&#39;s about having a clean career, if that makes any sense.</p> <p>I now put a much greater stress on my fellow engineers than I used to, and I care more about the team as a whole. In a lot of ways, this book takes the practices and techniques of many other books on this list and codifies them into an over-arching set of guiding principles. Software Craftsmanship as a movement can get a little culty at times, but I generally consider myself part of that cult, and I largely have this book to blame. The night time is the right time.</p> <p>What&#39;s especially great is this book is it&#39;s been licensed under Creative Commons, and is now <a href="http://chimera.labs.oreilly.com/books/1234000001813/index.html">completely free on the web</a>! Cool!</p> <p><strong>What it changed:</strong> How I view my responsibilities as a professional, and what I consider my true title.</p> <h2><a href="http://www.amazon.com/Art-Agile-Development-James-Shore/dp/0596527675">The Art of Agile Development</a></h2> <table class='image alignright' width='234'><tr><td><a href='http://www.amazon.com/Art-Agile-Development-James-Shore/dp/0596527675'><img src="http://www.nomachetejuggling.com/assets/theartofagiledevelopment-228x300.jpg" width='228' height='300'/></a></td></tr></table> <p>The first job I had out of college was pure chaos. No process, no estimation, no planning, nothing. Generally someone from marketing would stop by a programmer&#39;s cubicle and inform them that they just sold a few thousand dollars worth of seats based on a feature that didn&#39;t exist yet, so how long would it take to implement it? Being my first post-college job, I was in &quot;sponge mode,&quot; so I simply thought this was how it worked in the real world. It wasn&#39;t until my next job that I was introduced to Agile Development methodologies by way of Scrum, which was like mana from heaven. I was hooked.</p> <p>My job after that was at a company that wasn&#39;t just into Agile as a methodology, their core business was actually developing agile tools for other software shops to use. The entire company lived and breathed agile, so knowing agile was the same as understanding the core company domain. It would have been impossible to do my job without understanding agile, in more ways than one. So when I first took the job, I decided I needed to read an Agile book to make sure I knew my stuff. Based on the title, I picked up <em>The Art of Agile Development</em>. What I didn&#39;t realize at the time was that there were a lot of different agile methodologies, and in fact this book wasn&#39;t about Scrum, it was about XP.</p> <p>I became a die-hard XP programmer without even realizing it. My first exposure to &quot;XP Programming&quot; was a failed experiment in college that ruined it for me, I never would have knowingly bought a book on XP. But The Art of Agile Development changed how I do my job, it changed the processes I like to use when working with managers and other developers, and the practices I like to adhere to myself, such as Test-Driven Development, Spiking, Evolutionary Design, and the like. What&#39;s ironic is that I read this book to work at an agile company, only to find most of them disliked XP, and considered themselves Scrum only.</p> <p>What&#39;s nice about XP is that it&#39;s pretty individualistic. You can employ XP principles as a developer while working within Scrum, Kanban, Crystal, Lean, or whatever else. In fact, that&#39;s exactly what wound up happening: a small contingent of developers at this company including myself began working in a more XP-style within the confines of the company&#39;s Scrum processes, and our successes wound up infecting larger and larger groups of people until pretty much the entire engineering team was working similarly. When the company switched from Scrum to Kanban, it had little effect on how we worked.</p> <p>Today, my preferred way of working is with XP-style practices within a Kanban-style process, and an enormous part of that is because of this book. I wish I had a Kanban book to recommend as well to round this part of my list out, but 100% of my Kanban experience was gained on the job, with no books of any kind. What&#39;s more, having worked for three years at a company where agile was something bordering on a religion, I&#39;m pretty burned out on the topic in general, so other process-centric books on my &quot;to-read&quot; list have found themselves migrated towards the bottom. Nonetheless, in all of my reading, The Art of Agile Development was easily the most influential book on how I like to work. This one is pretty subjective, as I&#39;m pretty sure <strong>any</strong> good XP book would have had the same effect, but this was the one that did it for me, so I had to include it here.</p> <p><strong>What it changed:</strong> How I like to work in terms of processes and practices.</p> <h2><a href="http://www.amazon.com/Domain-Driven-Design-Distilled-Vaughn-Vernon/dp/0134434420">Update: Domain-Driven Design Distilled</a></h2> <table class='image alignright' width='234'><tr><td><a href='http://www.amazon.com/Domain-Driven-Design-Distilled-Vaughn-Vernon/dp/0134434420'><img src="http://www.nomachetejuggling.com/assets/domaindrivendesign.jpg" width='228' height='303'/></a></td></tr></table> <p>When I first posted this list, I gave an honorable mention to <a href="http://www.amazon.com/Domain-Driven-Design-Tackling-Complexity-Software/dp/0321125215">Domain-Driven Design</a> (DDD) even though I had never read it. My rationale was that, I had a hunch that once I finally read a book about Domain-Driven Design, it would have a career changing impact on me, but unfortunately I didn&#39;t like the style (or size) of the book itself, so I gave the honorable mention. Basically I wanted a book that gave me an overview of Domain-Driven Design without being a 560-page reference book. I wanted the &quot;Head First Design Patterns&quot; to the original Domain-Driven Design&#39;s &quot;GoF&quot;. Something to make the material easier to digest more quickly.</p> <p>I also said I had hoped <a href="http://www.amazon.com/Implementing-Domain-Driven-Design-Vaughn-Vernon/dp/0321834577/">Implementing Domain-Driven Design</a> (IDDD) would be that book (so much so that I pre-ordered it), but it seemed to assume the reader has already the original DDD book, and it was even longer than the original DDD book, clocking in at 656 pages. That&#39;s over 1,200 pages about domain modeling, it just seemed like massive overkill.</p> <p>Finally, Domain-Driven Design Distilled, by the same author as IDDD, was released. At a mere 147 pages, the stated goal of DDD Distilled was to get the information into the brains of as many people as possible without overwhelming them, give the basics and an overview of DDD in an approachable manner, and then allow readers to go deeper with DDD or IDDD later on. This was precisely what I was looking for and the book absolutely delivered - I really loved it.</p> <p>I kind of always had this hunch that Domain-Driven Design was something of a buzzword fad, that it likely described something I was already doing regularly and that the book and the approach likely just lent formality and terminology to common sense activities. After all, the biggest thing I see referenced seems to be this Ubiquitous Language stuff, which I think just means using the same nouns for stuff as the domain experts, which I try to do anyway so I&#39;m sure I&#39;m already doing everything in the book, right? Nope. I was flat wrong, which is why I consider this book a must-read for engineers who do a lot of greenfield work, domain modeling, and architecture. </p> <p>Early on, the author provides a sort of toy example that will stay with us for the duration of the book, designing the domain for a Scrum management product. I&#39;ve actually worked a job where I did this very thing, so this resonated fairly strongly. The book suggests that, if engineers are left to their own devices, they&#39;ll design around code generality to reduce duplication, so there might be like a ScrumElement that could be a Product or a BacklogItem, and there&#39;s like a generic ScrumElementContainer which could be either a Sprint or Release. I&#39;m just reading this section like, yeah, that&#39;s exactly what I would do... in fact I did that. Is that bad? But the rest of the book explains exactly why that&#39;s bad, and exactly how to do it better. Chapter after chapter, the book showed me the ways in which my approach to domain modeling was disastrously bad and how much better it could be. It also explained how, with this alternative approach, my domain would lend itself more easily to modular system design along service-oriented boundaries.</p> <p>In short, this book is excellent and completely changed how I think about and model domain objects at work. The book can sometimes be light on detail, I often found myself wanting more information, or stronger examples of exactly how something should work, but at the end of the day that&#39;s the purpose of this book - a short introduction that encourages the reader to dive in deeper with Domain Driven Design or Implementing Domain Driven Design. As such, I can&#39;t really complain about the general lightness of this book, as it&#39;s the primary reason it was such an easily-digestible 147 pages.</p> <p>Overall, this book is a must-read, I wish it existed years ago. I think back to all the times that a group of coworkers and I would gather in front of a whiteboard and model domain objects together without a single domain expert in the room. It makes me slap my head at how idiotic my approach has been for over a decade, the ways that I let database and technical concerns dictate the design of domain objects rather than the business&#39;s needs. I can never look at this regularly-performed process the same way, which is why it joins my list of Career-Changing Books.</p> <p><strong>What it changed:</strong> How I approach domain modeling; how I design service and module separations</p> <h1>Honorable Mentions</h1> <p>There are a number of books that I didn&#39;t include in the above list, but that nonetheless had a large impact on my career. This, of course, despite the fact that I completely cheated in my Top 10 and included more than ten books. </p> <ul> <li><a href="http://www.amazon.com/Presentation-Patterns-Techniques-Crafting-Presentations/dp/0321820800"><strong>Presentation Patterns</strong></a> - Only an honorable mention because it&#39;s not <em>really</em> about software development per se, but I&#39;ve <a href="http://www.nomachetejuggling.com/2013/06/10/book-review-presentation-patterns/">written elsewhere</a> about how this book changed my life and I have to include it somewhere in this post.</li> <li><a href="http://www.amazon.com/Pragmatic-Thinking-Learning-Refactor-Programmers/dp/1934356050"><strong>Pragmatic Thinking and Learning</strong></a> - Learn more about your brain than you ever realized you needed to know. Though not specifically about programming, it&#39;s a very programmer-centric view of the mind, and how one can best work with your own mind and improve your ability to think and learn.</li> <li><a href="http://www.amazon.com/Effective-Java-Edition-Joshua-Bloch/dp/0321356683"><strong>Effective Java</strong></a> - I said I wasn&#39;t going to include any technology-specific books, but I can&#39;t help but mention <em>Effective Java</em> somewhere. I was programming in Java for years before reading this book, but afterwards I felt like a Java master. I almost never work with pure Java anymore, instead largely using other JVM-compatible languages, but the Java I wrote before reading <em>Effective Java</em> looks very different than the Java I wrote afterwards, and I definitely prefer the latter.</li> </ul> <p>So that&#39;s my complete list. I obviously have many, many more books to read, and I look forward to writing another list like this one in the future after being profoundly changed for the better some more.</p> <p>Have some books you want to add? Feel like telling me one of my favorite books is inferior to one of yours? Want to yell at me for not including <a href="http://www-cs-faculty.stanford.edu/%7Euno/taocp.html"><em>The Art of Computer Programming</em></a> (come on, you never read that shit and you know it)? Leave a comment!</p> Wed, 05 Feb 2014 00:00:00 -0500 http://www.nomachetejuggling.com/2014/02/05/top-10-career-changing-programming-books/ http://www.nomachetejuggling.com/2014/02/05/top-10-career-changing-programming-books/ CenturyLink: Traps and Caps <p><em>TL;DR: After a decade of service, CenturyLink decided I wasn&#39;t worth keeping as a customer, so I switched to Comcast Business Internet. Even if CenturyLink tells you that you have no data caps, you do.</em></p> <p>When I first moved to Colorado nearly a decade ago, I signed up for Comcast&#39;s Residential high-speed internet service, and I hated it. I had to reset my cable modem once every week because it would stop working properly, and my internet noticeably slowed down when people would get home from school/work in my apartment complex.</p> <p>I did some research and determined that, for the kinds of internet speeds being offered at the time (around 1-5Mbps down), Qwest&#39;s DSL had similar prices to Comcast. Additionally, because it was DSL, I could use my own Netgear router that had a DSL modem built in, and I wouldn&#39;t have to mess with company equipment. Also, DSL&#39;s speed varies based on your distance to a hub, not based on how many people are currently using it. I thought I could avoid both of my major problems by switching to DSL for about the same price, so I did, paying for the maximum possible speed at the time, 5Mbps.</p> <p>For many years, Qwest was the only monthly bill I never had any trouble with. Sometimes my cable would go out or look pixelated, sometimes my heating bill would be surprisingly massive, I was always having annoying issues with my cell phone service, but month after month, I never even noticed I had Qwest. It was just there, it worked, it never went down, it never slowed down. It was great.</p> <table class='image alignright' width='306'><tr><td><img src="http://www.nomachetejuggling.com/assets/Qwest-300x233.jpg" width='300' height='233'/><p class='caption'>The good ol' days.</p></td></tr></table> <p>As the years went by, Cable speeds seemed to be improving at a faster rate than DSL speeds. Nonetheless, I was happy with Qwest&#39;s customer service and I had bad experiences with Cable, so I stuck to DSL. One day, I got a mailer that 10Mbps was now available in my area. I called Qwest that day and upgraded. A year or so later, another mailer came with 20Mbps as an offer, and I upgraded immediately. Each time the price difference was less than double for double the speed, so it seemed worth it. I use a lot of internet, and I still missed my University&#39;s T3 line.</p> <p>Eventually Qwest became CenturyLink after an acquisition/merge. I was a bit worried at first, I was happy with Qwest not only from a technical standpoint, but from a customer service standpoint. Would this change? I could always tell that Qwest representatives were especially friendly to me after they saw how long I&#39;d been a customer, I feared I&#39;d lose my &quot;standing&quot;. But, CenturyLink one day sent me a letter offering &quot;fiber&quot; internet, 40Mbps down, 5Mbps up. Upgrading bumped my price up to about $100 a month, which was a lot, but I really do use the everloving crap out of some internet, so I did it.</p> <p>Many people on Twitter and Facebook asked me about internet providers, and I&#39;d always talk up CenturyLink. Fast internet, good customer service, no downtime. And best of all, this was around the time that cable companies started introducing usage caps on their cable internet. I found this maddening for both philosophical and technical reasons, but <strong>CenturyLink had no such caps, so I&#39;d often argue it was a better choice for heavy internet users like myself</strong>. I know that I influenced a handful of people to get CenturyLink, or even in a few cases to switch to CenturyLink off Comcast.</p> <h1>The First Warning</h1> <p>In February of this past year, I was at work and decided I wanted something off my computer at home. I tried to SSH into my workstation at home but was unable to connect. This was strange. Like I said earlier, CenturyLink never went down, so it made no sense to me. Thinking maybe my workstation had crashed or died, I tried connecting to my home VPN, and that didn&#39;t work either. There are a few services I run on various machines that sync with centralized servers, such as <a href="http://www.subsonic.org/pages/index.jsp">Subsonic</a> and <a href="http://www.plexapp.com/">Plex</a>, so I tried connecting to them, only to discover they too had lost connectivity to my home. I even tried pulling up my <a href="https://nest.com/">Nest</a>, and saw that even that had lost connectivity to the outside world.</p> <p>Here&#39;s how much confidence I had in CenturyLink: <strong>I actually thought the power must be out.</strong> I went home at the end of the day, fully expecting to walk into a cold house that had been without power for a portion of the day, and assuming I&#39;d have to go around resetting clocks and booting up computers. But to my surprise, the house was fine. All the clocks had the right time on them, the power had never gone out. It was an internet problem.</p> <p>I sat down at a computer and opened up a web browser. Instead of my home page, I got this:</p> <table class='image aligncenter' width='586'><tr><td><img src="http://www.nomachetejuggling.com/assets/centurylinkuse-580x391.png" width='580' height='391'/><p class='caption'>No 'net for you!</p></td></tr></table> <p>This was why none of my devices were connectable. Every outgoing connection was pulled into some weird walled garden with this message, and I wasn&#39;t able to get my internet back until I clicked Continue to acknowledge I&#39;d gotten the warning. </p> <p>Like I said, and like I had told many people, CenturyLink <strong>had no usage caps</strong>. I had never heard of this Excessive Use Policy before, I didn&#39;t remember getting a letter saying one had been implemented. I used quite a lot more internet in January than usual, but I had no idea it was something I needed to curb. </p> <p>Totally baffled by this, I called the number in the notice. I asked what the policy was and the tech told me to hold on while she looked up my account. Alright, I thought, this is good. Generally as soon as they look up my account, they see how long I&#39;ve had service with the company and their tone becomes super-friendly. Sure enough, when she came back on the line, she was quite friendly. <strong>She told me that, because I pay for the highest tier internet the company offers, 40Mbps down, that the Excessive Use Policy does not actually apply to me.</strong> She said the notice was in error, I wasn&#39;t supposed to get it. She said she&#39;d flip a flag to make sure I never get another notice again. I thanked her and asked her to please make a note of this conversation in my account, so that it would be there in case it happened again and I had to call back. She said she would, but reiterated that it was not necessary, as it would not happen again.</p> <p>When I first got the warning message, I posted it to my Twitter, expressing some irritation. A bunch of friends asked me what happened, since I was always talking up CenturyLink. After I got off the call, I posted again with an update, that it was all good, and I wasn&#39;t even supposed to get the warning. I publicly praised CenturyLink&#39;s customer service yet again.</p> <blockquote class="twitter-tweet" lang="en"><p><a href="https://twitter.com/CenturyLink">@CenturyLink</a> resolution: I&#39;ve been removed from the list of accounts that get such notices. Good on you guys, thanks Kensie. <a href="https://twitter.com/search?q=%23happycustomer&amp;src=hash">#happycustomer</a></p> <p>&mdash; Rod Hilton (@rodhilton) <a href="https://twitter.com/rodhilton/statuses/301883017645928449">February 14, 2013</a></p></blockquote> <p><script async src="//platform.twitter.com/widgets.js" charset="utf-8"></script></p> <blockquote class="twitter-tweet" lang="en"><p><a href="https://twitter.com/CenturyLink">@CenturyLink</a>, my tweet about how well you resolved this issue has 2 of my friends considering switching from Comcast. :)</p> <p>&mdash; Rod Hilton (@rodhilton) <a href="https://twitter.com/rodhilton/statuses/301907519121530881">February 14, 2013</a></p></blockquote> <p><script async src="//platform.twitter.com/widgets.js" charset="utf-8"></script></p> <h1>The Second Warning</h1> <p>I resumed business as usual, confident that the usage policy did not apply to me, happily using my internet like crazy. But a few months later, in May, I was working at home and, once again, I was presented with a similar walled-garden message. The first time I was surprised and alarmed, but now I was annoyed. I had already discussed this with a representative, and she assured me I wouldn&#39;t get another one of these.</p> <p>I called again, and hid my irritation, knowing that if I stayed friendly, as soon as the technician pulled up my account, they would see how long I&#39;ve been a customer and be friendly as well. Once again, the tech pulled up my account notes, and was immediately accommodating. I asked if my notes indicated I&#39;d called about this before, and the representative told me <strong>yes, he could see that I had spoken to the company a few months ago, and was told I was not supposed to get these.</strong> The representative put me on hold so he could call some other department and be certain that the proper flag was set so that I wouldn&#39;t get any more of these. After about 10 minutes, he came back and told me it&#39;s done, he&#39;s sure I won&#39;t get any more of these. I asked him to please make another note in my account that we had this conversation, and he said he did.</p> <p>Once again, I live-tweeted most of these interactions. And once again, people responded to it. One thing that kept popping up was, people advised me to look into Comcast Business Internet. They said it was more reliable than Comcast Residential, and had absolutely no data caps because it&#39;s for businesses. Curious, I looked into it, even going so far as calling a Comcast Business representative to ask some questions and get a plan priced out. I was surprised to see that the price for Comcast Business Class internet was about the same as what I was paying, for slightly faster access. All this time, I&#39;d been comparing Qwest/CenturyLink&#39;s price increases for more speed, never really comparing with competitors. </p> <p>Comcast Business looked interesting, but in the end, I didn&#39;t like the idea of having to sign up for a year or two of service ahead of time with a contract, and I didn&#39;t fully trust Comcast due to my experiences with the residential cable. The price of Comcast Business Internet was good, but the only real advantage was the removal of data caps, which I&#39;ve now had two CenturyLink representatives tell me don&#39;t apply to me anyway. My faith in CenturyLink restored, I once again went about my business.</p> <blockquote class="twitter-tweet" lang="en"><p>2nd Excessive Use warning from <a href="https://twitter.com/CenturyLink">@CenturyLink</a>. Called and they said (again) I&#39;d never get another. Love the customer support.</p> <p>&mdash; Rod Hilton (@rodhilton) <a href="https://twitter.com/rodhilton/statuses/334521053403549697">May 15, 2013</a></p></blockquote> <p><script async src="//platform.twitter.com/widgets.js" charset="utf-8"></script></p> <h1>The Letter</h1> <p>I never got another walled-garden notice. Months went by, and I figured that the extra effort the CSR took to get me flagged properly finally worked, and I was in good shape. Then, on November 21st, I got a letter. Not a browser notice, not an e-mail, but a paper letter, sent via the post office. The last 3 times this company sent me something in the mail, it was to alert me of higher speeds in my area. This letter was decidedly different.</p> <table class='image aligncenter' width='586'><tr><td><img src="http://www.nomachetejuggling.com/assets/centurylinktermination-e1386560161832-580x576.png" width='580' height='576'/><p class='caption'>You're Terminated.</p></td></tr></table> <p>So many things annoyed me about this letter. The claim that this was my third notice was questionable, to start. Though I had gotten two previous warnings, both of them were in error according to CenturyLink customer service. So as far as I was concerned, I had gotten zero warnings. Additionally, the termination date was December 3rd, eleven days away. Eleven days to find a new internet provider, get signed up, schedule an install, get everything working? Infuriating.</p> <p>Once again, I held back my irritation, knowing that being friendly was always met by helpful representatives once they saw how long I&#39;d been a customer. I called the number on the letter, which connected me to technical support, and person who I will call Jordan. <strong>Jordan checked my account and said that my account was not flagged for termination, had no excessive use flags on it, and was in good standing.</strong> He called the letter a &quot;miscommunication&quot; and said I would not have my service ended. This was good news, so I relaxed a bit. I did, however, ask if he saw the two notes on my account from previous calls about similar issues, and he said that there were no notes on my account. That made no sense to me, the second time I called, the person I spoke with specifically mentioned seeing the note from the first time I called, so how could it be gone? Jordan informed me it might be a &quot;business note&quot; and I should call the business office on Monday, because he only had access to &quot;technical notes&quot;.</p> <p>I called the business office on Monday using a number Jordan gave me and spoke to a second CSR, who I will refer to as Taylor. Taylor verified two things for me. One, I absolutely did not have any notes on my account about previous phone calls. And two, <strong>my account was just fine, I was not set to be terminated, I had nothing to worry about</strong>. Again, I relaxed some more, but then Taylor suggested we conference-call someone from the &quot;Securities Office&quot; just to triple-check. She got an employee, who I will refer to as Howard, on the line.</p> <p>Howard did some checking and gave me some news that stunned me. <strong>The excessive use policy <em>does</em> apply to my account, and I am indeed flagged for termination on the 3rd, now in eight days.</strong> I was almost at a loss for words. I was told to ignore both warnings by CenturyLink employees previously. I called the number <em>on the termination notice</em> and was told my account was fine, and even followed up with a second call to double check it. Even that call said I was fine. It wasn&#39;t until I talked to a third person, fifth overall, that I was finally told for the first time that the policy applied to me, and I was being told not only that but that as a result my service was being terminated in eight days.</p> <p>I finally got the details on the Excessive Use Policy from Howard. He told me that the limit for usage is 250 GB per month for any service at 1.5Mbps and up. Anything above that was a violation of the policy. I wasn&#39;t sure I understood, so I asked for clarification, &quot;you mean that someone who is paying for 40Mbps down and someone paying for only 1.5Mbps down have same usage cap?&quot; and Howard answered, &quot;correct.&quot;</p> <p>&quot;So even though I&#39;m paying more than quadruple what my 1.5Mbps neighbor is, he and I have the same maximum?&quot;</p> <p>&quot;Yes.&quot;</p> <p>I did a quick calculation. 40 megabits per second is 5 megabytes per second. 250 GB is 256,000 megabytes. That means, if I were to saturate the bandwidth I&#39;m paying for, I could use up my entire allotment in 51,200 seconds. 14.2 hours. I asked Howard if this was the case and he informed me that it was.</p> <p>This was about the time I decided to stop being so friendly. I explained that it was ridiculous that I was having my service terminated in eight days when I&#39;ve gotten, as far as I&#39;m concerned, zero actual warnings that were legitimate. Howard explained that the CSRs I talked to previously were misinformed, as the policy does apply to me. He also said I had no notes about previous conversations in my account. As I mentioned, what&#39;s particularly surprising about that is that the second CSR mentioned the notes of the first, meaning that, if they were gone, they must have been deleted (or the second CSR lied). Amazing, too, that two independent CSRs would tell me the same thing incorrectly.</p> <p>I asked Howard to reset my &quot;warning&quot; counter back to zero, so I at least had a fair chance now that I knew the policy was in place. He was willing to reset me back to two. So my internet would no longer be terminated in eight days, but if I violated the policy one more time, I&#39;d get another letter of termination. In other words, <strong>I was now in a permanent position of possibly having my internet suddenly turned off within about a week</strong>.</p> <p>Howard suggested I look into CenturyLink Business Class, which removes the cap. He hung up, and Taylor (who was still on the line) conference called someone from the Business Internet department. The Business CSR informed me that the price would be about $200 a month, double what I was paying at the time. I asked if it would at least be faster, and was told it would not be, 40/5 is the fastest CenturyLink gets. I tell him (and Taylor) I need to think about that a little more. The Business CSR hangs up, which unfortunately disconnects the entire call somehow.</p> <p>Ugh. I still had questions for Taylor, she said she would stay on the line after the Business call. I call back and get a new CSR, who I will call Brandon. I asked to be transferred to Taylor because she&#39;s been informed of my extremely long story, and I didn&#39;t want to have to repeat it all. Brandon informed me they are not able to transfer to people directly, only departments. I let out a sigh and once again recounted the entire story starting from my first warning up until that very moment. I explained to Brandon that it doesn&#39;t seem right to give me a data cap so low that I can bust out of it in 14 hours, and it seems that the caps should, at the very least, increase linearly with the speeds. Brandon agreed with me, agreed that I&#39;m not getting proper treatment, and asked to put me on hold while he gets to the bottom of this. <strong>While on hold, I was disconnected again</strong>.</p> <p>At this point I was in full-on pain-in-the-ass-customer mode. I called again, asked to speak to Taylor or Brandon, and was once again told they can&#39;t do that. So I asked to speak to the highest manager I can be transferred to, because if I have to tell the whole story again I want it to be at the highest level I can get. The CSR obliged, transferring me to someone I will call Terry. I told the entire story again. By the time I&#39;m done, I&#39;ve been on the phone for three hours in total. All I wanted at this point is to be reset to zero warnings so that I could try to get my usage down to acceptable levels without living in fear of termination. Terry told me he can&#39;t really help me as a Customer Support person, and offered to transfer me to Securities. Fine.</p> <p>I was transferred to Securities again and spoke with a Securities Agent. She looked up my notes and informs me that Howard already did me &quot;a big favor&quot; by resetting me back to 2 warnings. She said there was nothing further she was willing to do for me. I asked how, exactly, I&#39;m supposed to even get my usage down to their limit when my next violation means termination. She told me there are free tools I can install on my computer to monitor usage. That&#39;s rich. I&#39;m supposed to install these tools on every computer in my house, both mine and my wife&#39;s, plus our phones, running the gamut of Linux, Windows, OS X, and Android? The best I can do is look at my router, which tells me how much traffic I used in a day, so I&#39;d have to check it at the end of each day and manually total it up. I started thinking about writing a shell script for this, and then chastised myself for playing into this nonsense. The Securities Agent once again recommended I switch to CenturyLink Business Class, all for the low-low price of double what I&#39;m paying now for the exact same service.</p> <h1>The Defeat</h1> <p>Completely defeated, I hung up, and I started looking at Comcast Business Class again. If I were willing to pay $200 for internet, I could get Comcast Business Class&#39;s fastest speed, 100Mbps down, 20Mbps up, with no data caps. Hell, at my current price of $100, I could get 50 down and 10 up, 10Mbps faster downloading, with double the upload speed, and still no cap.</p> <p>I called Comcast and got some more info. I even had a contract drawn up, ready to sign. At this point, I couldn&#39;t justify staying with CenturyLink any longer. I could remove the data caps from my life with no additional cost just by switching providers, and I would even get a bonus of slightly faster speed. I obviously had reservations about Comcast due to my previous experience, but everyone I was talking to on Twitter indicated that the Comcast Business is a completely different branch of the company, that the techs really know their stuff and that the representatives are extremely helpful, because they are used to dealing with businesses.</p> <p>Speaking of Twitter, I was naturally live-tweeting everything that was happening with all of this. In addition to friends and colleagues responding, I actually got a response from CenturyLink itself. </p> <blockquote class="twitter-tweet" lang="en"><p><a href="https://twitter.com/rodhilton">@rodhilton</a> Hey Rod, our team can help look into that for you. Send us your acct info here <a href="http://t.co/V7SCkTtg1e">http://t.co/V7SCkTtg1e</a>&#10;Thanks, ^AshleyKay</p> <p>&mdash; CenturyLinkHelp Team (@CenturyLinkHelp) <a href="https://twitter.com/CenturyLinkHelp/statuses/405002367793254400">November 25, 2013</a></p></blockquote> <p><script async src="//platform.twitter.com/widgets.js" charset="utf-8"></script></p> <p>After some more tweeting, I got this as well:</p> <blockquote class="twitter-tweet" lang="en"><p><a href="https://twitter.com/rodhilton">@rodhilton</a> Hi Rod, if you need additional help with this issue we can have our DSL Escalations Team give you a call&#8230;Thanks,Corey</p> <p>&mdash; CenturyLinkHelp Team (@CenturyLinkHelp) <a href="https://twitter.com/CenturyLinkHelp/statuses/405045978928926720">November 25, 2013</a></p></blockquote> <p><script async src="//platform.twitter.com/widgets.js" charset="utf-8"></script></p> <p>So I decided to bite. Maybe this is a different team, who might really be able to do something for me. Again, all I wanted at this point was to be reset to zero warnings so I had a fighting chance.</p> <p>I filled out their form, writing 1,414 words explaining this entire story. I explained that Comcast had them beat on Business Tier service by a wide margin, and I&#39;d leave for Comcast before paying double for the same service from CenturyLink just to avoid being harassed. I also explained that this is my last attempt to remedy the situation with CenturyLink, that I&#39;d given them three hours of my Monday and I couldn&#39;t give them any more of my time. I reiterate I&#39;ve been a customer for a long time, and I&#39;d like to stay a customer, but that I needed this situation to be improved.</p> <p>I submitted the form and waited. I had a contract with Comcast Business Internet ready to sign in my inbox. If CenturyLink&#39;s help team could do something for me, I&#39;d keep them and try to figure out how to get my bandwidth to reasonable levels. If they wouldn&#39;t do anything for me, I&#39;d switch to Comcast, severing nearly a decade of loyal service. </p> <p>That night, I checked with my router to see what my bandwidth use for the day was. Just a normal day of working from home, no big media downloading, no movie or music streaming, nothing excessive. I used 10GB that day alone, and that&#39;s with trying my hardest not to use much. At that rate, I&#39;d hit the cap in 25 days, less than the month I&#39;m paying for. While I was tweeting about all of this, I had some friends ask me just what the hell I&#39;m doing that&#39;s using so much bandwidth. To be completely honest, I don&#39;t even understand the question. After seeing how much bandwidth I use on a day when I&#39;m giving it my full effort to minimize use, I have to wonder just how the hell everyone else is able to stay under their caps at all. It seems impossible.</p> <p>A few days later, I got a response from CenturyLink.</p> <blockquote> <p>Thanks for reaching out and sending such candid feedback on your experience. I understand your frustration, unfortunately there&#39;s no way I can change or stretch our policy. Up until this year you haven&#39;t hit over 250gb before in, as you mentioned, many years. You have since gone over the usage limit every month so far in 2013. We&#39;ve been very generous in the notices you&#39;ve received and when you&#39;ve received them. I apologize for the 2 representatives that provided you with incorrect information and therefore caused further turmoil in the matter. I am investigating that further so that proper feedback/coaching will be enforced. I wish there was something I could offer as far as a better price goes for the business usage, but there&#39;s no other option other than what you&#39;ve been told already. Please let me know what direction you&#39;d like to go with this and I&#39;ll take care of you to the end. If there&#39;s anything else additional I can help with, let me know as well.</p> <p>I wish there was a way for me to guarantee that you&#39;ll go back to &#39;zero&#39;, but that&#39;s not an option. As Howard already did for you, the forgiven process is the best than can be done. I understand your decision to try other venues for service you need. If there&#39;s anything else I can assist you with, please let me know.</p> </blockquote> <h1>The Switch</h1> <p>A Comcast technician installed my new Business Internet two weeks later. I know this post is way too long already, and I don&#39;t want to look like some kind of shill for Comcast, so let me just summarize with: <strong>why didn&#39;t I do this sooner?</strong></p> <table class='image alignleft' width='306'><tr><td><img src="http://www.nomachetejuggling.com/assets/comcast-business-300x168.png" width='300' height='168'/><p class='caption'>The good new days</p></td></tr></table> <p>We did a speed test before he left, and I was getting 57 Mbps down, 11.5 up. I&#39;ve had no issues with bandwidth dropping at all, it seems pretty consistent all day long; the lowest I saw was 48 down, which is still faster than my top speed with CenturyLink (about 37). I haven&#39;t had to reset the cable modem at all, it&#39;s a beefy, boxy beast of a thing from Netgear. No frills, no wireless, just a black box I hooked up to my existing router.</p> <p>I&#39;ve had no blips, no lags. I&#39;ve worked from home a few times since the install and I&#39;ve seen no drop-offs, disconnections, or slowdowns. I&#39;ve dealt with Comcast Business Internet representatives in the sales department, billing department, and technical support department, and all of them were extremely helpful and friendly. The Technical Support guy even helped me fix an issue I was having with my router, even though it wasn&#39;t Comcast-commissioned equipment.</p> <p>I actually feel bad for how many people I&#39;ve recommended go to CenturyLink. I know a nontrivial number of people have signed up with them based on my glowing recommendations over the years (this isn&#39;t self-absorption, they&#39;ve told me this). I think I&#39;d now only recommend residential CenturyLink to very light internet users, like my parents or in-laws. There&#39;s no good reason at all to use CenturyLink&#39;s business class service, which is massively overpriced for the service when compared to Comcast&#39;s offering. And the higher-speed tiers at CenturyLink are all hampered by data caps that can be blown out in less than a day. If all you want is 1.5-5Mbps down for light internet use, CenturyLink is the way to go for price, but for higher speeds it&#39;s a complete rip-off. The same high speeds can be found for cheaper at Comcast Business, with no data caps.</p> <p>It&#39;s unfortunate that things had to become so unpleasant with CenturyLink but it became clear that the company simply was not interested in keeping me as a customer. While I wish I could have left CenturyLink on different terms, I&#39;m glad I made the switch, and I hope the quality of service with Comcast Business Internet continues.</p> Tue, 17 Dec 2013 00:00:00 -0500 http://www.nomachetejuggling.com/2013/12/17/centurylink-traps-and-caps/ http://www.nomachetejuggling.com/2013/12/17/centurylink-traps-and-caps/