On Space and Success

On February 22nd, 2024, a two-ton robotic spacecraft called Odysseus made its final descent to the surface of the Moon, attempting to become the first American space vehicle to do so since the final Apollo mission in 1972. The operator, a Houston-based company called Intuitive Machines, had the most difficult part ahead still, and yet they had persevered through tremendous adversity already. Its arrival at the Moon was off-target, prompting the teams to fire up their laser range finders to confirm their altitude in orbit. The range finders failed, a possible death knell for a mission that depended on them for precision landing. Thinking quickly, they called up their main customer NASA and asked to use one of their onboard payloads, a similar laser technology, to incorporate it into their landing software and replace the missing data. NASA agreed, and Intuitive Machines wrote a daring on-the-fly software patch just an hour or so before landing and uploaded it to Odysseus.

Intuitive Machines

If you were hoping for a software engineering triumph, you'll be disappointed. A misconfigured setting meant the NASA data was not properly ingested into the landing guidance system. Even worse, the team didn't know it at the time, so they began their descent thinking they had just saved the day. Odysseus came in hot, reaching the surface of the Moon about three times faster than expected and with a one metre per second horizontal velocity. One of its landing legs caught a rock and snapped apart, tipping the lander over on its side as it came to a full stop.

The moments after were tense in the control room. Back on Earth, we did not know what had happened yet; controllers were stuck waiting and listening for a call from the vehicle. The call was late. It took fifteen minutes to finally get a carrier signal and give everyone enough confidence that Odysseus was at least intact. NASA, who was hosting a live video stream, quickly declared success and shut down the feed, and we all returned to our lives and waited to hear more.

In the coming days we would learn that the vehicle had indeed tumbled on its side. The radio, pointed down at the Moon instead of sideways towards Earth, was weakly bouncing signals off its surface, making it difficult to hear. Solar panels were not ideally pointed, resulting in reduced power and forcing a shortened mission. It's even possible that the steep slope of the landing site, a last minute change from their main customer NASA, had complicated things.

And yet, Odysseus had indeed landed soft(ish)ly, breaking a long drought for American spacefaring organizations on the lunar surface. Many payloads had successfully beamed back some kind of data. And the operators were learning what worked, and what didn't, in order to feed forward their new expertise to the next mission, coming as soon as this year. Both NASA and Intuitive Machines declared victory while its lander lay hobbled on its side, a broken leg, weakly calling home before the cold lunar night silenced it forever.

A seemingly easy question to answer was suddenly not. Was IM-1 a success?

Pedantry

In the Off-Nominal Discord server (a great space to hang out with a bunch of space geeks while simultaneously supporting all the work my partner Anthony and I do) we have a fun prediction-based game best known for embroiling us in vociferous debates about space events. The gist of it is that users can make predictions about things that will or won't happen in the world of space, and when we finally learn the outcome, meaningless internet points are awarded or subtracted. The outcome is adjudicated via public voting, and so you can imagine that the text of your prediction is important to get right and can be the source of great discussion. And it just so happened that a couple months before this landing, one member made one such prediction:

Intuitive Machines IM-1 / Nova-C lander mission will successfully land on the moon in 2024.

The word "successfully" became a point of interest. No one disagreed that it had landed. But did it land successfully? Some argued the old aviation line "any landing you can walk away from is a good one" applied. Others (including me) pointed out that it could hardly be called successful given it failed to read its altitude, came in sideways, broke a landing leg, tipped over, and caused a premature end of mission. There was some fun discussion on both sides. In the end, the Anomalies out-voted me by a 21-17 margin. This is tight by the standards of the game, since most predictions are pretty clear to adjudicate.

It's not the first time this has happened. The very first vote that stirred significant debate goes all the way back to March 2021, when SpaceX launched the Starship prototype labeled SN10 (Serial Number 10). The flight reached ten kilometres in altitude before it descended back to Earth for a vertical landing. With a bit of a fire up one side, the vehicle touched down with a little bounce and a slight lean. Everything seemed great until about 8 minutes later when the entire ship exploded. A similar prediction about its landing had been made, with even more, shall we say, fluid language.

SN10 will stick the landing

LabPadre Space

What does stick the landing even mean?! It had touched down softly, sure, but something about that operation had caused the vehicle to explode! Was this a success? If so, how much of a delay between touchdown and explosion was permissible before it wasn't? These are nuanced questions! For what it's worth, I voted "no" on this one too, but was outnumbered 14-11. I guess the Anomalies are a positive bunch.

Complexities

Meaningless internet points aside, this idea of success has actually been something I've been thinking about a lot over the years covering space. There are so many examples of missions where opinions on success are spread across a spectrum. Half of InSight's two primary payloads failed, and yet it gave us an unprecedented dataset describing the interior of Mars. The NASA Commercial Crew program has spent billions of dollars on Boeing's Starliner vehicle without realizing a single astronaut delivery to the Space Station, and yet it also enabled one of the safest and inexpensive means of transportation ever in SpaceX's Dragon V2. The James Webb Space Telescope was delivered a decade and a half late and $9B over budget (yes, that's a B after that figure), and then absolutely nailed the most complex engineering deployment sequence ever imagined by a satellite designer, giving it a long life of game-changing science discoveries ahead of it.

As I thought more about these missions, the idea of complexity kept presenting itself to me in different ways. Space missions, like any large engineering project, are complex on multiple axes.

An obvious type of complexity, which I'll call Technical Complexity, is when the project has many capabilities or parts that can succeed or fail independently. NASA's planetary missions almost always fit this bill. The Curiosity Rover, exploring Mars since 2012, has over a dozen scientific instrument suites. There are seventeen total cameras on board. It has six wheels, a robotic arm, a mast, a power supply, redundant computers and communications methods, and thermal regulation. This kind of complexity means that even if individual parts fail, like how its wheels are shredding, or its computer was acting up, or its drill feed motor seized, the mission can continue and be successful. This leads to all kinds of questions about the definition of success. How many instruments would it take to fail before we called Curiosity a failure? Does it matter if they fail early or later in the mission? Are there some components that have a bigger weight on success than others?

There's another kind of complexity that I'll call Objectives Complexity. I tend to define this as outcomes-based success at a single mission level. Sometimes these are really straightforward, like on one of Blue Origin's New Shepard tourist flights. These short duration launches take six passengers to just over 100km in altitude then parachute them back to Earth in the span of about 10 minutes. Your basic objectives are "take people to space" and "bring them back safe". But in other cases there can be incredibly complex objectives, split into tiers and with sub categories of success. The Curiosity rover's successor Perseverance has a really deep science traceability matrix, which you can find in this 154-page science definition report. Add in that these kinds of questions take years or decades to answer, science objectives are incredibly difficult to adjudicate, especially when they come in groups like this. How many science objectives does Perseverance have to publish papers about before it's considered successful? What if it takes decades to do so? What if it fails to answer its major questions but makes an unexpected discovery? What if it answers them all but with null results?

NASA

Up next you have Program Complexity. We're looking now beyond a single mission and looking at objectives of an overarching program as it proceeds through its missions. This is one that I think is often completely overlooked and lost when news agencies try to cover space. Headlines are generally not great at conveying Program Complexity while still enticing you to click. Billionaire Elon Musk's taxpayer-funded rocket explodes again will get you a lot more clicks than reporting on the rapid pace, trending successes and technical breakthroughs of a multi-decade program like SpaceX's Starship. But you don't even need to look at a polarizing figure like Musk to find examples of this, and it's not always a good thing being reported as bad. Sometimes it's the reverse. NASA's Artemis 1 mission, a success on almost all counts, is still a joyful flashpoint in a program that is horrifically slow, technically unsound, exorbitantly priced and ultimately incapable of realizing the lunar program dreams of the space enthusiasts watching it.

Lastly, a final layer of complexity is something I am calling Stakeholder Complexity. Often, these kinds of projects are large enough that they cannot go forward without a coalition of support from multiple stakeholders. NASA has been juggling this kind of complexity for its entire existence; they're charged with fulfilling their duties under the NASA charter, but they have to do so in a way that benefits the Executive Branch, hundreds of congress people who write its budget while representing hundreds of districts, company lobbyists, and of course, all 300 and whatever million tax-paying Americans for whom they fly. And heck, let's throw in the benefit of the rest of Earth and humankind in there, too, since everyone is always touting that line around. Each stakeholder defines their own success based on their own reasons for being a part of the mission, and their judgement of success or failure can and probably will be different. The International Space Station is a great example of something with a tremendous amount of Stakeholder Complexity. It's everything I described above multiplied by five space agencies and 15 countries. You could write books about perspectives on the ISS's success or failure as a program.

As you can see, answering this question is not easy.

Simpler times?

I asked myself if the recent spate of discussions on complexity and success were something new or if it had always been there. As I am prone to do, I looked back through history to see what had been said about these kinds of things.

Of course, all of the complexities I've described have been present since day 1 of space exploration. Look no further than the Apollo Program, a success in space history that will never be forgotten, and yet one that faced many criticisms. Apollo was incredibly expensive, killed three to six astronauts (depending on how you count them) plus nearly 3 more, and straddled the United States with a massive amount of ground infrastructure whose costs have been weighing it down and muddying engineering decisions ever since. This is not to mention the entire program's roots tracing back to a difficult and confusing origin in Nazi Germany. Was the V2 program successful? Have fun answering that one.

Back in December, my friend Casey Dreier did a great episode of the Space Policy Edition of Planetary Radio discussing the Space Shuttle as a policy failure. They covered everything from the inability to reach its cost goals, a confusing mix of stakeholders between NASA, the Department of Defence, and the regular congressional interests, and a technical design by committee that created such a strange looking (and behaving) launch system. But they also consider the shuttle instrumental role in building the International Space Station. The podcast uses as a starting point a critical paper by space historian John Logsdon, which was published in 1986.

A maybe lesser known event that is difficult to adjudicate is the Atlas ICBM program, a frantic military endeavour meant to develop a rocket system capable of delivering a ballistic nuclear weapon to the other side of the planet during the Cold War. It faced enormous political challenges, funding issues, technical problems and more, and after finally securing the contract and producing the Atlas SM-65, it was made obsolete almost immediately by the Titan missile. Atlas never truly did what it was designed for, but it made incredible technological breakthroughs and went on to be converted to an orbital rocket that eventually gave way to the Atlas V, a vehicle still flying today and one that has sent a space probe to every planet in our solar system. Not to mention, we got WD-40 out of it!

Convair/General Dynamics

So I think it's safe to say that the struggle to define success amid high complexity has been around a while. But I still feel like something is different.

Commercial Space

That last complexity type, Stakeholder Complexity, had me thinking a lot. One thing that is different about the present age was that the commercial stakeholders, the private companies out there building and operating these missions, are playing a much larger role than before. In the early days of the space programs, governments ran the show. There was stakeholder complexity built in because, well, governments are complex, but because of the nature of politics, a lot of the underlying stakeholders get abstracted away. Ask 100 Americans which space agency landed people on the Moon and they'll all say NASA. Then ask them who built the lunar lander they did it in and you'll hear a lot of silence (it was Grumman, by the way).

Today this just isn't true; NASA has been making huge efforts for the last 10-20 years to acquire space services rather than operating them themselves. It's the difference between buying a car to drive to work vs taking a taxi. NASA doesn't wanna own the car anymore. This has been great because it's given rise to lots of new companies like SpaceX and Blue Origin and of course, Intuitive Machines, who are all spinning up new space services that they can sell to NASA and others. When NASA next lands on the Moon, everyone will know who built the lander. They'll be able to tell you who the CEO is, too. These companies are major stakeholders now, and they're going to be defining success in their own ways.

As private companies are want to do, they're innovating to increase revenue and drive down costs. This has created some interesting tension in places where previously there wasn't much. Take for example, the way these services are developed.

In past eras, spaceflight was a new and risky endeavour, and to justify its cost and risk, the public needed a lot of assurance that safety and reliability were being taken seriously. NASA's Mission Control coined the now famous phrase "Failure is Not an Option", which perfectly encapsulates this way of thinking. Build it, test it 1000 times, and be as sure as you can that it is going to work. That was the only way. This testing process, known as V&V (Verification and Validation) is incredibly expensive and is a main cost driver for NASA engineering projects.

Private industry has taken a different approach to reliability - fly a lot, fail early and iterate. The old Von Braun quote is "one test is worth a thousand expert opinions". No one exemplifies this better than SpaceX, who takes a whimsical approach to blowing stuff up in the name of engineering. And it works - the Falcon 9 rocket today is objectively the best launch vehicle system ever made. But this kind of bold approach to failure is not always easy to adjudicate. And it's not just SpaceX doing it. In an excellent interview on Main Engine Cut Off, Intuitive Machines co-founder and CTO Tim Crain told a story about the old NASA project called Morpheus that highlights this perspective shift (Morpheus technology went on to become integrated into Odysseus).

So we had the Morpheus Alpha vehicle and we had a hardware failure on the first free-flight of it; we lost the vehicle. We had the Bravo vehicle about 60% complete on that project, because we were pushing on that project so fast that we had decided (and got the Agency [NASA] to agree) that the loss of the first vehicle was not a mishap...the risk and the price was acceptable for moving as fast as we did with Morpheus.

At its face, it's absolutely wild to consider any loss of vehicle situation as anything but a mishap. Yet, Tim's point here is not without merit, because now that technology is on the Moon. These anomalies might be mission failures, but they can lead to program success for the company, and with companies being major stakeholders now, this matters a lot more.

NASA

The way they market their services is changing too. Doing more with less is the name of the game in capitalism, and for a rocket that means putting more satellites in the payload bay. Ridesharing, where many smaller payloads fly together and deploy at once, is a growing business in spaceflight. This equivalent of a bus with a rocket engine is an efficient mover of satellites, as evidenced by SpaceX's Transporter series missions and NASA's Commercial Lunar Payload Services (CLPS) mission, of which Intuitive Machines is a part of. This way of flying to space directly increases Stakeholder Complexity...by literally adding stakeholders! When SpaceX's Starship comes online we're bound to see some truly huge ride shares as they try to fill up the roughly 1000 square metres of space in the nose.

Odysseus

So this brings us back to Intuitive Machines and its IM-1 mission. It's a textbook example to demonstrate how much complexity there is in a modern space mission, and one that makes it incredibly challenging to distill down to a single true or false in the success column. But by thinking about the different types of complexity, I think we can take a stab at coming to some kind of result.

Thinking about IM-1's technical complexity, there are some wins in spite of its failures. The failure of the laser range finder was a tough break that caused many of the downstream issues in the mission. But it wasn't a death knell like they previously feared, and the mission continued, albeit in a limited fashion, despite this issue. The mission also faced some early problems with its star trackers and was tumbling after separation from the rocket, but the teams were able to overcome these. IM-1 also demonstrated some great technological wins. Achieving the landing they did on just their inertial measurement unit and some optical navigation is remarkable. They proved out their communications model and demonstrated the use of a methane engine in space, an important breakthrough. On balance, I think Odysseus showed some incredible robustness and I'm giving this mission a success on technical grounds.

Moving to objectives complexity, IM-1 faced some difficulties but did meet some of their objectives. Thinking about payloads, NASA says that they received some data from five of their payloads, though its hard to imagine that that data was all acquired in a nominal experiment scenario, given the orientation of the lander and the shortened duration. They have yet to test a sixth payload. This may not matter from Intuitive Machines' perspective, however. In the recent earnings call on March 21st, the company reported that they expect to earn 95% of the award payments from NASA, which might indicate that NASA was pretty lenient on the quality of data returned. In other words, NASA may have made the award payments contingent simply on getting any data back, rather than meeting a specific success criteria. Another non-NASA payload, EagleCam from Embry-Riddle Aeronautical University, could not be deployed on descent as expected but was deployed after landing, then failed to communicate back to the lander. Other payloads were more passive, having "succeeded" just by virtue of being on the Moon. On a technical objectives level, Intuitive Machines made some great achievements, including launch, transit navigation, orbital insertion, and landing with sufficient precision. The company was hoping for a touch down two to three kilometres from the target, and achieved 1.5km. They then continued to operate on the surface, transmitting data back, though in a limited fashion. I'm giving IM-1 a mostly successful on objectives grounds.

As we move up the complexity tree, things get harder to adjudicate, and in some cases we don't have enough information one way or another. At a program level, we have two things to think about. The first is the company's Nova program, or more broadly its lunar delivery business. When trying to adjudicate success here, I tend to ask myself a simple question: was IM-1's outcome helpful or harmful for Intuitive Machines? It's hard to say without being internal to the company, but in listening to Tim Crain speak, I believe they are optimistic and that the lessons learned are going to feed forward into an even better IM-2 mission. Financially the company seems well-positioned. Despite the volatility around a lunar crash landing, the stock price is up well over 150% YTD, and in the most recent earnings call the company reported healthy cash levels.

The second program level consideration is NASA's Commercial Lunar Payload Services program. Was IM-1's outcome helpful or harmful to CLPS? The program is still in its infancy, having just this year finally realized its first two launches after its conception in 2018. The first, from company Astrobotic, suffered a failure in space and never made a landing attempt. CLPS really has two objectives at a program level. The first is to deliver science from the Moon. So far, it has only partially succeeded at that, with whatever data they were able to scrounge up from these two missions. They have yet to nominally operate a payload on the surface. The other objective is to foster a commercial market for lunar payload delivery. We're frankly way too early to make a call on that one. We've got companies giving it a go, but we don't yet know if that market exists, and if it does, we don't yet know if this model is best suited to capitalize on it.

One thing I do want to bring up about program feasibility is a point made by Eric Berger in another interview with my friend Anthony on the Main Engine Cut Off podcast.

Do you think what they [Intuitive Machines] did with this first mission is more complicated than what SpaceX was trying to do with Falcon 1? Because SpaceX had a similar amount of money, and it was about the same time frame (about 4 years) where they built the Falcon 1 and launched it for the first time. Same size company...

Anthony and Eric both agree that the Moon landing was harder, and so do I. Talking about the price of the mission (NASA awarded Intuitive Machines a total of $118M for this flight), it's a pretty remarkable feat to have developed, tested, launched and operated a lunar lander for that much money, especially considering it included the development of a methane rocket engine. The company of course had outside funding as well, but from a CLPS program perspective, that's a pretty good value generated for NASA's investment, and gives me hope. We just need to see if the model sustains it.

At a program level, therefore, I'm giving IM-1 a success rating of "wait and see".

Lastly, we have stakeholder complexity to consider. This idea is partially explored in the program level discussion above (since Intuitive Machines and NASA are the primary stakeholders), but we can also consider the remaining stakeholders, like the customers themselves. These are probably the most mixed results. The NASA payload science teams operating instruments are probably disappointed, even if they understood the risks. I don't know if Columbia sportswear got whatever data it needed to test out its fancy insulation tech, but the shot of their logo over the Moon should pay back their investment in marketing alone. The guy with 125 Moon Balls got his art to the Moon, so that one seems pretty straightforward. I'm giving this one a partial success, given that the NASA payloads were the primary anchor customer.

Conclusion

Alright, so given my perspectives above, you can see that where I arrived is that Intuitive Machines is generally successful. So what's the point? Did I really just lay out a whole blog post to get to the same place as most journalists covering this event got to a month ago?

I'm a big believer in nuance, which makes me a big hater of headlines. I get a lot of joy from digging deep into a topic and coming to greater, multi-dimensional understandings of them. As someone covering space and playing in this domain, I get asked a lot to distill these events down into successes and failures, and it's something I always have and always will shy away from. Things are complex, and things have uncertainty, two concepts that do very poorly in a tweet. I feel comfortable in that complexity and uncertainty, and I think you should too.

Unless of course meaningless internet prediction points are on the line, in which case be sure to have a fierce, axiomatic and unchangeable position!