A Case of Extreme Uptime

Ongoing maintenance of our machines is important. You should schedule change windows and make sure servers have the latest firmware and OS patches. Performing regular maintenance is the simplest way to avoid security vulnerabilities. Keeping current on fixes can save you from calling IBM Support; often their first response to a question is to tell you that your issue has been resolved in an already released service pack.

That said, AIX systems provides us with world-class technology. These machine are capable of running for a very long time without any care. And a few do.

A friend recently forwarded this email concerning one of his AIX machines:

 

            I have a production server that was here when I started in 1999. It was last booted on Jan. 14, 2000, almost 13 years ago...

            It was renamed after applications were migrated off of the server two weeks ago. It is now going to be used as a DR box. As you can see below, it was up 4,675 days before I rebooted it this morning. And yes, it came up just fine.

 

            # oslevel -r

            4330-11

            # uptime

            09:35AM   up 4675 days,   2:21, 2 users, load average: 1.22, 1.29, 1.28

 

This is, of course, first and foremost a tribute to the quality of AIX systems. However, a not insignificant amount of good fortune is also involved. This box ran continuously for almost 13 years. Power outages were never an issue. Any hardware issues were resolved through hot swapping. No one accidentally logged into this production server and accidentally ran a shutdown –Fr. The firewall that this box must have operated behind kept it safe from constant attacks.

I was impressed to hear of a production AIX server running for this amount of time without even a reboot. I imagine there are systems that have been up even longer, though I couldn't find anything specific. If you'd care to do your own research, there are threads devoted to this sort of thing. See here, here, here, here and here.

Frankly, I wouldn't recommend treating a machine this way. I always want to be sure I'm running a supported operating system with the latest fixes. Still, these types of stories surface every now and again, maybe you have your own. What's the longest-running production system that you know of? What were the circumstances? Please share your anecdotes in Comments.

Comments

[Anthony English]

The oslevel reported 4330-11, in other words AIX 4.3.3 Maintenance Level (ML) 11. That was the latest level for that AIX release, so they were patched up after all.

Not sure how they did it, though, as ML 11 came out in Feb 2003 - 3 years into their 13-year uptime, and I expect it would require a reboot after patching.

Resilient.

[Sebastian]

Hi
We have another similar system:
root@invisip1:/ > uptime
08:39AM up 3724 days, 16:10, 1 user, load average: 0.00, 0.01, 0.01
root@invisip1:/ > oslevel -r
4330-10
Two yeas ago it was productive, now it only runs for the uptime counter, lots of errors in the errpt, I think this system would not boot again if it gets a shutdown.

[Hubba8Bubba]

One of our customer's (print office) p520 uptime was about 8 years. But the power down in a whole village made UPS turn it off as it couldn't work any more. I only switched one of the disks in rootvg after that.
PS: AIX 5.3.

[Marcelo]

Be careful with the information presented by the "uptime" command. If you shutdown xntpd and change the system date let's say 20 years back then reboot and after that change the system date to the present, the uptime command will report 20 years of uptime.

Cheers.

[johann]

I would feel as a criminal if I had to reboot such a system :)