agile

What’s new in GNU make 4.1?

October 2014 saw the release of GNU make 4.1. Although this release doesn’t have any really remarkable new features, the release is notable because it comes just one year after the 4.0 release — that’s the least time between releases in more than a decade. Hopefully, this is the start of a new era of more frequent, smaller releases for this venerable project which is one of the oldest still active projects in the GNU suite. Read on for notes about the new features in GNU make 4.1.

MAKE_TERMOUT and MAKE_TERMERR

Starting with 4.1, GNU make defines two additional variables: MAKE_TERMOUT and MAKE_TERMERR. These are set to non-empty values if make believes stdout/stderr is attached to a terminal (rather than a file). This enables users to solve a problem introduced by the output synchronization feature that was added in GNU make 4.0: when output synchronization is enabled, all child processes in fact write to a temporary file, even though in effect they are writing to the console. In other words, the implementation details of output synchronization may interfere with behaviors in child processes like output colorization which require a terminal for correct operation. If MAKE_TERMOUT or MAKE_TERMERR is set, then the user may explicitly direct such commands to maintain colorized output despite the fact that they appear to be writing to a file.

Enhanced $(file) function

The $(file) function was added in GNU make 4.0 to enable writing to files from a makefile without having to invoke a second process to do so. For example, where previously you had to do something like $(shell echo hello > myfile), now you can instead use $(file > myfile,foo). In theory this is more efficient, since it avoids creating another process, and it enables the user to easily write large blocks of text which would exceed command-line length limitations on some platforms.

In GNU make 4.1, the $(file) function has been enhanced such that the text to be written may be omitted from the function call. This allows $(file) to work as a sort of “poor man’s” replacement for touch, although having reviewed the bug report that resulted in this change, I think this is more an “enhancement of convenience” than a deliberate attempt to evolve the program. Of course I have to give a shout out to my friend Tim Murphy, who filed the bug report that led to this enhancement — nice work, Tim!

Relaxed constraints for mixing explicit and implicit rules

The final feature change in GNU make 4.1 is that make will emit a regular error rather than a fatal error (which terminates the build) when both explicit and pattern targets are specified as outputs of a rule, like this:

1
foo bar%: baz

This is an interesting change mostly for the high level of drama surrounding it. That bit of syntax is clearly illegal — in fact, if the pattern target is listed first rather than the explicit, GNU make has long identified this as invalid syntax, terminating the parse with *** mixed implicit and normal rules. Stop. Unfortunately, due to a defect in older versions of GNU make this construct is not prohibited when the explicit rule is named first.

In 3.82, the GNU make maintainers fixed the defect: whether or not the explicit target is named first, GNU make would identify the invalid syntax and terminate parsing. Everything was fine for about a year, and then? People flipped out. As it turns out, this construct is used by a prominant open source project: the Linux kernel. The offending syntax had been eliminated from the main development branch shortly after the 3.82 release, but third-party developers suddenly found themselves unable to build legacy versions of the kernel with the latest release of GNU make. A bug report was filed and generated 21 reponses, when the average GNU make bug report has only 3. Ultimately, the maintainers relented by reducing the severity to a non-fatal error for the 4.1 release — but with a stern message that this will likely become a fatal error again in a future release.

Bug fixes and thoughts

In addition to the bigger items identified above, the 4.1 release includes about two dozen other bug fixes. Overall, this release feels like a minor one — as often happens when release frequency increases, the individual releases become less interesting. From an agile/continuous delivery standpoint, that’s exactly what you want. But I’ve found that it is also difficult for a team that’s accustomed to less frequent releases with larger payloads to transition to smaller, more frequent releases while still incorporating large changes that take longer than one release to implement. Of course, one point does not make a line — that is, we can’t tell from this release alone whether the intention is to switch to a more frequent release cadence, or whether this release is an exception. If they are trying to increase the frequency, I think it will be very interesting to see how the GNU make development team adapts to the new cadence. Regardless, I’d like to congratulate the team for this release and I look forward to seeing what comes next.

SPARK 2013 by the Numbers

SPARK 2013
A few weeks ago we wrapped up the sixth annual Electric Cloud Summit: SPARK 2013. This year’s event was hands down the best we’ve done, with more content, more speakers and more attendees than ever before. For the first time we had invited keynote speakers including agile development and continuous delivery luminaries like Jez Humble (who literally wrote the book on continuous delivery!) and Gene Kim. We also had live streaming so that people who couldn’t make it to the conference in person could still watch and listen to the keynote sessions — if you missed the conference, you can watch the recordings now, and I really recommend that you do.

As usual, I did some analysis of the event once the conference was over. Here are the results.

Registration and Attendance

Each year since its inception, the summit has set a new record for total registrations, and SPARK 2013 was no exception with 186 people signed up. But even more impressive is the record 168 attendees — those people that actually made it to the conference. That beats the previous high of 146 from 2011 and is a massive 33% increase from the 126 attendees in 2012:

SPARK 2013 Attendees

But that’s not the end of the story on attendance this year, because for the first time we offered live streaming over the Internet. That added an impressive 84 additional “virtual” attendees to the keynote session, bringing the total to over 250 attendees.

I think three factors contributed to the high registration and the better-than-90% conversion rate. First, there’s no doubt that the list of keynote speakers helped attract people to the event:

  • Jez Humble, co-author of “Continuous Delivery”
  • Gary Gruver, co-author of “A Practical Approach to Large-Scale Agile Development”
  • Gene Kim, author of “The Phoenix Project”
  • Paul Rogers, Chief Development Office at General Electric

Second, this was the first time that the conference was open to the public rather than being exclusively for Electric Cloud customers. Finally, this was the first time that attendees paid to attend the conference — somewhat counter-intuitively, you can sometimes increase interest in an offering simply by charging more for it. I think this has to do with the perceived value of the offering: some people think, “If this is free, it must not be very good.” Plus, once you’ve paid for a conference, you’re more likely to attend because you don’t want your money to go to waste.

Repeat attendance

A solid 25% of the attendees in 2013 had attended at least one previous summit, slightly down from the percentage of repeat attendees in 2012, but in line with the historical average. Amazingly, three die-hard users have attended all six conferences!

SPARK 2013 Repeat Attendees

Presentations

SPARK 2013 had about 20% more sessions than 2012, and again more of the content came from users and partners than in any previous year. Sadly I didn’t get a chance to see too many of the presentations since I was a presenter myself, but I did get to watch the keynotes at least. If you didn’t watch the SPARK 2013 keynotes yet, you really should. It’s OK, I’ll wait.

The 2013 conference had 35 sessions in total, spanning four days and three tracks, including all the keynotes, training and track sessions:

SPARK 2013 Presentations

Origins

As usual, the majority of attendees were from the United States, but a respectable 10% braved international travel to attend in person:

SPARK 2013 Attendee Countries

Fourteen US states were represented — the exact number of states represented in 2012 and in 2011, but a different set from either of those years. If I didn’t know better I’d say this was evidence of some kind of conspiracy. As expected, most of the US attendees were from California, but about 20% were from other states:

SPARK 2012 Attendee States

Industries and Delegations

67 companies sent people to SPARK 2013, representing a broad array of industries. Some of those are the usual suspects, like software and telecommunications, but there are some surprises as well, like the 4 companies in the retail industry and the one in education. As they say, software is eating the world. Many companies sent only one representative, but just a bit more than half sent two or more. One large networking company sent fifteen people to SPARK 2013!

SPARK 2013 Industries

Rate of registration

Finally, here’s a look at the rate of registration in the weeks leading up to SPARK 2013. In 2012 I hypothesized that the relatively low attendance numbers were partly because promotional activity for the event didn’t really get started until about 9 weeks prior to the conference. I thought perhaps that was not enough lead time for people. But to my surprise, the same is true this year and yet we had significantly more registrations than in 2012. I still think we could get even more if we started promoting the event earlier, but obviously there’s more to the story than simply that. The good news is that the team behind SPARK 2013 is already planning for SPARK 2014, so hopefully next year we’ll see if I’m right.

SPARK 2013 Registrations

Don’t miss SPARK 2014!

Overall I think SPARK 2013 represents a turning point in the evolution of the Electric Cloud Summit. In a way it’s like we’re finally “growing up”, going from a small, private event to a serious public conference. I can’t wait to see what SPARK 2014 looks like, and I hope you’ll all join me there next year!

“Playing” with agile

Recently we invited a Scrum coach to Electric Cloud to teach us how to get started with the Scrum model of agile development. On the first day we played a game intended to introduce us to the core elements of Scrum: plan, do, inspect, adapt (or “plan, do, check, act”; or “the Deming cycle”). Without getting into a deeper discussion of Scrum itself, I thought I would share my team’s performance in this fun little game. If you’re familiar with ElectricAccelerator, our game strategy will come as no surprise: it exploits parallel processing and horizontal scalability to improve performance.

The game was simple: we were given a bucket of rubber bouncy balls and instructed to pass balls from person to person, until every member of the team had touched the ball. For each ball that completed the circuit we earned one point; for each drop we were penalized three points. A few rules made the game more interesting. First, it was forbidden for two people to touch a ball at the same time — there had to be “air time” between individuals. Second, we could not pass balls to the person directly to our left or to our right. Finally, there was a time limit (just like a sprint): we had only 2 minutes to pass balls in each round.

At the start of the game, we were given 5 minutes to plan our strategy and make a prediction of how many balls we would pass. Between each round we had 3 minutes more to modify our strategy based on our experience in the previous round and make a new prediction for the next round. If you are familiar with Scrum you’ll recognize the analogy to story points.

In total we had 12 players plus one scribe (me) that was tasked with counting the number of balls passed and dropped.

Round 1 (plan: 0; actual: 29)

Our first planning phase was best described as chaotic. It wasn’t actually clear who was on our team or not, due to some stragglers to the activity. We weren’t sure about the constraints. Everybody had ideas about how best to pass the balls, so everybody was talking at once. It seemed simple, but in fact we had barely gotten everybody in place when the 5 minute prep time elapsed. We did manage to agree on the three key elements of our strategy though:

  • Dropping balls into the cupped hands of the receiver, rather than throwing them, to minimize the risk of dropping balls.
  • Two rings of players, one inner and one outer, facing each other. Balls would be passed in a zig-zag between the rings.
  • Parallel passing. Everybody would be either passing or receiving at all times.

This diagram shows the positions of the players, as well as which players had a ball at the start of the round:

Scrum game, round 1

As you can see, we had too many balls “in play” when we started, given our strategy — a direct consequence of unclear communication during the planning phase. The surplus balls were dropped almost immediately. Our final score for this round was 29: 1 point for each of 35 balls passed, minus 6 points for 2 balls dropped.

Round 2 (plan: 50; actual: 72)

Round 1 demonstrated that our core strategy was sound, but to improve performance we decided to make a couple tweaks. First, we made certain that we were in agreement about which players would start with balls: only those in the outside ring. Second, we realized we could improve throughput by passing two balls at a time, instead of just one. With our drop-into-cupped-hands strategy this was hardly more risky than one ball at a time. We predicted that we would pass 50 balls, about 60% more than we did in round 1. Here’s the updated diagram showing the starting positions of the players and balls for round 2:

Scrum game, round 2

Our score in round 2 was 72: 72 balls passed, with zero dropped.

Round 3 (plan: 120; actual: 60)

At this point we believed we had everything worked out. We increased the balls-per-pass to three and predicted that this would result in about 120 balls passed. But during the planning phase one of our players abruptly left — to be honest I’m not even sure who it was or why they stepped away. All I know is that suddenly we had only 11 players instead of 12, which left us with 6 on the outer ring but only 5 on the inner ring. We didn’t realize the problem until people started lining up in position near the end of the planning period. With the clock ticking we made an exceptionally poor decision about how to handle the mismatch: one of the inner ring players would serve as receiver for two of the outer ring players. First they would receive from player A, then pass to player B; then immediately receive the balls back from player B before sending them on to player C. Sounds complicated, right? It was. Here’s the updated diagram:

Scrum game, round 3

This proved was disastrous for our performance. At speed, it was (understandably) hard for the player serving double-duty to efficiently execute the elaborate sequence of exchanges. In addition, we were careless when we grabbed the extra balls we needed: although most were consistently round, a few were those oddly shaped rubber rocks which move in unpredictable ways. These misshapen lumps of rubber are just a bit harder to catch than regular balls, and that slowed us down. Our final score in this round was just 57: 60 balls passed, one dropped.

Round 4 (plan: 120; actual: 123)

The obvious problem in round three was the mismatch in the sizes of the inner and outer rings. The solution was obvious too: remove one player from the outer ring to restore equilibrium. There was just one problem. According to the rules of the game, a ball had to be touched by every player in order to count as having been passed. What could we do? We pled our case to the coach, who agreed to let us have one person sit out this round — a demonstration of another fact of agile development: sometimes a team can be made more productive by having fewer people on it. With 5 players on each ring, we again predicted that we would pass 120 balls. Here’s how the layout looked for this round:

Scrum game, round 4

This was our best round yet with a final score of 123: 135 balls passed, with only four drops.

Review

Overall I was really pleased with our performance in this game — granted, the point of the exercise was not actually to see how many balls we could pass around, but to experience the plan-do-inspect-adapt cycle directly. And we certainly did that too. But come on! How can you not be excited by a more than 4x improvement in throughput from round 1 to round 4? I’m not surprised though. After all, speed is the name of the game for ElectricAccelerator. This is what we do. That we got there by applying the same strategies to this game that we use in Accelerator itself — icing on the cake.

Later that night I realized an error in our execution on round 4 though. We chose to even out the rings by dropping one player from the outer ring, when we could just as easily have added a player to the inner ring: me. As scribe, I did not actively participate in the ball passing, only the planning and review. But there was no particular reason I couldn’t have stepped in. That would have increased our throughput by 20% (by increasing the number of balls in play from 15 to 18). I think we could have exceeded 150 balls passed with that configuration. So in the end, the game was a great demonstration of what is probably the most important concept from Scrum: there’s always room for improvement.