Practical Technology

for practical people.

April 25, 2003
by sjvn01
0 comments

Getting Closer to 99.9999% Network Uptime

Back in May 2002, Cisco Systems announced new software for its 12000 series routers, Globally Resilient IP (GRIP). GRIP is meant to eliminate data loss on the network even if there are circuit failures or human errors. Not a bad trick if they can do it.

Juniper Networks, of course, immediately replied that said they have had “zero packet loss” on their routers since 1998. Without making that boastful claim, Alcatel had announced that its non-stop ACEIS router would be out later this year.

What is going on here? Has information loss become so crucial? Is there a real demand for highly available networks? Or, are the network OEMs, fighting against a horrid economy, simply creating a high-tech version of the washing powder commercials of my early 1970s childhood when Tide or Bold would roll out their “new-improved” products.

Some of it is smoke and mirrors. Sorry Juniper, no one always gets a perfect score in packet exchanging. As for Cisco and Alcatel? They won’t have product out until the 3rd quarter. Other companies, better known for their work with ATM, such as Nortel, and Terabit switching, Avivi, are also moving into the extreme reliability market.

Is there really a market for IP networking that strives to reach, and surpass, 99.9999% reliability and uptime? As a matter of fact there it is, but it’s not the traditional Internet, or even extranet or intranet markets. It’s the carrier market that supplying the demand for previously unheard of uptimes for IP.

In the past, IP has always been a best-effort networking solution. IP would work on anything-even carrier pigeons (RFC-3043)-but while the message more often than not would get through, sometimes your connection would be slow, sometimes it would be fast, and sometimes you’d wonder if the backbone provider really was using carrier pigeons.

Because of its low cost and increase in reliability, IP has become more and more attractive to all carriers though, and not just ISPs and LAN administrators. So it is that, according to David Willis, Meta Group’s VP of Global Networking Strategy, that there really is “clearly a need for more reliable IP network. Carriers are more migrating more and more of their traffic to IP traffic. AT&T carrying more and more voice on their IP backbone.”

In the past, even backbone providers and major ISPs have used the slogan, “every router needs a buddy.” What that meant for those who don’t spend their times in American Network Operations Centers is that every router had a hot spare running beside it. That was fine, if when a failure finally happens your customers could live with minutes or hours as the stand-by routers built up its route tables and took over the network load.

When your customer is Cable & Wireless, Equant, Infonet, or WorldCom, though, minutes don’t cut it and hours are completely unacceptable. An acceptable outage time in SONET for voice, according to Seamus Crehan Senior Analyst, Dell’Oro group, is a sub-15millisecond recovery. Why such a harsh standard? Because breaks in voice traffic directly impact voice-carriers’ bottom line.

Unfortunately, according to Tim Smith, an analyst for Gartner Dataquest, IP that can handle that kind of load isn’t here yet. Willis agrees, “This wave of carrier-grade IP services is new movement. It’s not fully ready yet.

So why are the carriers making this move? Cost. IP, even when running on the highest of high-end routers, looks to be cheaper than SONET and ATM in the long run. As Val Oliva L2/3 Product Marketing Manager for Foundry Networks and a member of the 10 GEA (10-Gigabit Ethernet Alliance) Board of Directors says, “Carriers looking for ways to increase profit, they will need a new set of systems that can lower their system cost and cost of operations to increase returns.”

And, sooner rather than later, IP will provide those new systems. Willis thinks, “Cisco, which always leads in anything related to core routing, will lead. Juniper, which had been taking market share from Cisco, but is now falling back, will also be in the hunt. Alcatel and Nortel, which already have strong carrier relations thanks to their ATM switching background, will also be in the race.” He also adds that any company that’s very active in MultiProtocol Label Switching (MPLS) will have a shot.

Another question in carrier-grade IP will be whether the future will be IP running on top of existing ATM and SONET networks or on Ethernet. “Ethernet,” You ask? Yes, Ethernet.

10-Gigabit Ethernet (GbE) will be emerging as a serious competitor for OC-192 SONET by the end of 2002. If anything, 10-GbE will be in place before high reliability IP routers become commonplace. In the event, both technologies are likely to support each other in advancing IP’s case to carrier-grade customers.

Only time will tell if Oliva is right when she says that, “Ethernet is IP’s best friend, and in the end, IP wants Ethernet.” What is certain is that IP is coming to the carrier world.

March 31, 2003
by sjvn01
0 comments

Mission Impossible? Stopping Spam

Your mission, should you choose to accept it, is to stop spam from overwhelming your bandwidth and entering your customers’ mailboxes. We have every faith that you and your anti-spam efforts will be successful. If not, your company will, of course, disavow any knowledge of your actions. This message will self-destruct in five seconds…Mission impossible? It used to be, but there are things you can do to stem the spam tide. And now, more than ever, is the time to strive to stop it. According to Ferris Research, a San Francisco and London-based e-mail and groupware analysis firm, finds that, “For U.S.-based ISPs, 30% of inbound email is spam, while at U.S.-based corporate organizations, spam accounts for 15% to 20% of inbound email. And that, “In 2002, the total cost of spam to corporate organizations in the United States was $8.9 billion.”Sounds unbelievable? Think again. Many ISPs think that Ferris’ numbers are on the low side. David Daniels, president and CEO of Starfish, a small North Carolina ISP reports that, “Our spam filters reject about six emails for everyone we accept.” He’s not the only one with awful experiences. James Triplett, CTO and founder of Thelix Internet of Amherst Massachusetts’s says that his company, “recently implemented a spam blocking solution, and was astounded to find that we are now blocking about 60% of the incoming traffic. 35,000 – 45,000 spam attempts per day, with about 10,000 messages accepted.

And, spam is only going to get worse. In May 2002, the Coalition Against Unsolicited Bulk Email, Australia did a survey showing that the amount of spam is doubling every 4.5 months. Spammers are also trying new ways to get mail into the Internet. J. D. Falk, member of the Coalition Against Unsolicited Commercial Email (CAUSE) Board of Directors says, “Forged headers are the (new) standard, because they still fool the vast majority of recipients.”

That only makes users hate spam more, but what many people don’t realize is how much spam hurts the bottom lines of ISPs. Alexis Rosen, president and co-owner of Public Access Networks, which runs Panix, the US’s second oldest still running ISP, finds that the “majority of incoming mail is spam. And, that, in turn, “chews up a lot of bandwidth and disk space.” But, what really eats up system resources and “puts significant stress on the mail server” is the constant disk I/O. “Disk activity is the most precious and expensive resource we have.” That, in turns, affects Panix’s, and every ISP’s bottom line.

How bad can it get? Daniels says, “In addition to the cost of bandwidth we have to run two mail servers. One does primary spam filtering and the second is a mailbox server. Running everything on one box was slowing down mailbox access even though the machine should be capable of supporting at least 10 times as many accounts as were on it. One of the techs or I will spend an hour or two a day answering customer spam complaints and updating Access Control Lists. The cost is substantial.”

That’s why, according to Marten Nelson, a Ferris Research analyst, “most ISPs use server-based (anti-spam) tools.” Nelson says corporate mail administrators have lagged behind ISPs. That’s because corporation e-mail addresses are less exposed to email address harvesting attacks. Ferris found that their spam load was more in the 20 to 25% range.

Corporate e-mail may be due to be hit by more spam. Mail harvesting attacks (MHA) in which a program attempts to send mail to a corporate domain and records which addresses don’t bounce are becoming more common. Making matters even worse, aggressive MHAs can also act as denial-of-service attacks.

To try to put spam back in the can, ISPs use a variety of products. Joyce Graff, a Gartner vice president and research director, believes that there are three primary approaches to fighting spam: hire a service, install an appliance or install layered software.

In a report on spam, she suggests “Smaller enterprises in particular (those with fewer than 5,000 employees) should seriously consider using a service.” Postini, one such service provider, uses a typical approach of intercepting and filtering all mail send via Simple Mail Transfer Protocol (SMTP) before it arrives at the gateway.

The most popular anti-spam program though, according to Graff, is, “Brightmail with the dominant share of the service provider market.” Unlike Postini, Brightmail is sold primarily in a product line that runs on Windows2000 and Solaris.

The other major commercial anti-spam players, according to Ferris are “MAPS, Trend Micro, and Tumbleweed. Cloudmark and MailFrontier are new entrants that have interesting approaches to fighting spam.”

Some companies, though, also like to build their own solutions. For example, Panix offers users a variety of anti-spam tools based on using filters built around the procmail Unix mail processing utility and SpamAssassin, a popular open source mail filtering program for Unix and Windows. There are other popular open source anti-spam products like Len Conrad’s IMGate, which acts as a SMTP filter sitting between mail servers and the Internet.

Regardless of the programs you use, you should supply the anti-spam programs at the server, not the client, level. While client solutions are very popular, they don’t help save your bandwidth or disk I/O and that’s where the re-occurring costs of spam bite any Internet providers the hardest.

It works well, but, as Conrad says, “there is no one ‘silver bullet’ to stop abuse. There is not even a dominant approach. The only ‘universal’ is that mail abuse levels are horrendous. We are at war.”

March 31, 2003
by sjvn01
1 Comment

Cyber Cynic: Bye-Bye SCO

What is SCO thinking?

Suing IBM for a cool billion pits them against one of their oldest allies and puts many of their resellers in the uncomfortable spot of having their OS vendor being at war with their hardware vendor.If you know me at all, you know I’ve been covering SCO and Caldera for over a decade and that, generally speaking, I’ve liked what they were doing first as a network administrator, systems analyst and developer and later as a technology journalist.But the Caldera/SCO I knew is, I’m convinced, as dead as a doornail.

This move has alienated all their Linux users and the rest of the Linux community. It’s also, according to the resellers I’ve spoken too, made their lives a lot harder. I mean how can you get someone to upgrade to SCO UnitedLinux when chances are they know that SCO has taken legal action that threatens the Linux community? You’d have to be a better salesman than The Music Man’s Professor Hill!

SCO’s broad claims about its Unix’s IP have made even OpenServer and UnixWare customers nervous. They know their SCO OSs safe, but many of the bigger ones have AIX5L, directly targeted by SCO, Solaris, or HP-UX installations working in the back room. They worry that eventually they’ll be the ones paying higher, much higher, license fees to their vendors thanks to SCO’s legal actions.

Do they think they can win? I doubt it. Boies, Schiller and Flexner is a top law firm, but I’ve been watching Unix intellectual property fights since Unix Systems Laboratories (USL, whose intellectural property is now owned by SCO) took on Berkeley Software Design, Incorporated (BSDI, now owned by Wind River) in 1992 and, if there’s one thing I’ve learned, no one, except Microsoft and other non-Unix OS vendors, wins these battles.

The actual result of that case, by the by, was that BSDI beat the initial USL injunctions and that after Novell bought USL, Novell CEO Ray Noorda famously declared that he’d rather compete in the marketplace than in court. SCO, although primarily owned by The Canopy Group, which is titulary headed by the ailing Noorda, has choosen to compete in the courts..

Now, I’m not an attorney, but I do know my SCO and Unix’s intellectual property history. So, yes, I do think SCO has some valid complains about IBM not pushing their joint project, AIX 5L on Itanium, as hard as they could. But, IBM did that though for the best of business reasons: Itanium was, and is to this day, a marketplace failure. I know IBM thought, and I agreed then and now that there wasn’t enough of a market for AIX 5L on Itanium to make it worth the time to produce it. Still, such is the stuff that business lawsuits are made of. Usually, however, the law suit comes when the conflict still has some life to it and AIX5L on Itanium has been moldering in the grave for over three years now.

And, in any case, that avenue is not the one SCO is pushing. Instead, their main thrust as SCO says in its complaint is that IBM misappropriated the confidential and proprietary information from SCO in Project Monterey” for Linux’s advancement.

I can’t take SCO’s greater claim that Linux has somehow benefited from IBM leaking System V Unix secrets via AIX into Linux, I can only ask what has SCO been smoking? IBM has always taken a hands-off approach to Linux. They’ve never wanted to develop it, they expected their Linux partners, Red Hat, SuSE, Turbolinux and, oh yes, Caldera/SCO to develop Linux. Simply contributing things such as printer drivers and a journaling file system, which may or may not have any connection with Monterey or AIX 5L to open source, as SCO claims, has nothing to do with

If indeed SCO Unix intellectual property has slipped into Linux, which I strongly doubt, then the most likely source is SCO itself. Indeed, when Caldera announced that they were buying SCO, in their press kit they included the following quote from one of my Smart Partner articles: “By making sure it doesn’t buy into the Unix business’ bad old tricks and staying a Linux company at heart, Caldera can earn a true victor’s crown. UNIX’s branding and technology, the SCO’s reseller channel–they all count for a lot. But, using the open approach to create a much broader market for all UNIX-like operating systems will be Caldera’s best move.”

In what is perhaps the most surprising thing of all about SCO’s suit, Caldera/SCO, itself, was until this suit one of the most important Linux distributors and developers makes it clear that Linux is now the enemy.What is this? In fact, Daryl McBride, SCO’s president and CEO, told me in a recent interview that SCO is “about business, not Unix. Linux and Unix aren’t different animals, they’re in the same pen, and they can run common applications.”

What’s really going on here? Well, I think SCO is moving out of the operating system business into the law suit business. In talking with the other Linux and major Unix players, no one, but no one wants to partner with SCO anymore. Their reseller partners aren’t happy either.

So, like a murder mystery, the question is who stands to gain something? A look at the stock market will show who benefits, at least in the short run: SCO’s long suffering stock holders. The day after the law suit was announced, SCO reached a 52-week high of $3.66. It’s dropped since then, but at its current (March 31th) range on the close order of $2.71, its still doing better than it has in ages.

I’ve also said though that I think it unlikely SCO will win the law suit, or any related ones they might launch in the future. And, I suspect they know it. So, what I see happening is that SCO’s ownership and new management has decided to try to make money by making the company an attractive take-over target.

Think about it, IBM could buy SCO, even at its current pricing, for what amounts to lunch money for Big Blue. Or, say Microsoft decides to buy it to entangle Linux and Unix companies in a tangled web of fear, uncertainty and doubt (FUD) that could last for years.

The net result? In the first case, Unix, Linux, and their companies are likely to continue on their way and SCO’s owners and new officers walk away with a pocket full of money. In the second, they still walk away, if not rich, certainly better off then they are now, and everyone, but everyone, in the Unix/Linux business is in for the worse time in their lives.

We can also hope that, one way or another, the law suit is settled quickly. IBM can afford a long law suit, Linux companies can’t afford to have customers always asking what will happen to their operating system contracts were SCO to actually win the law suit.

In any case, the SCO we’ve known, loved and hated, partnered and worked with, is gone. The name may be the same, but that’s all that is. SCO resellers can hope and pray that someone buys out OpenServer, UnixWare, SCO Linux Server and the reseller channel who’s willing to work with them, instead of simply junking them to get SCO out of the market.

January 9, 2003
by sjvn01
0 comments

SCO Linux 4 – Ready for the Big Time

SCO, formerly Caldera, have taken the lead in bringing UnitedLinux consortium’s UnitedLinux server operating system to the reseller market. And, that’s big news. SCO Linux 4, SCO’s version of UnitedLinux 1.0, may not look that different from OpenLinux. In fact, it really just looks like a typical-albeit stripped down to the vital server basics-Linux server distribution. Which, when you get down to it, is exactly what it is. But, that’s the point

UnitedLinux is an attempt to create a standard business server Linux with common file directory conventions, command options, installation routines and high-end options like clustering and shared memory multiprocessing (SMP). The main idea behind UnitedLinux is that when a customer buys a UnitedLinux branded distribution he can be certain that any UnitedLinux applications will run on it without tweaking.

As resellers know, common business application and operating system compatibility is far more important to customers than having the latest and greatest file system. So it is that SCO Linux 4 has more in common with business operating systems like Windows 2000 Server or Solaris 9, than well-thought of, but end-user oriented, Linuxes like Debian or Slackware.

That’s not to say SCO Linux 4 isn’t really Linux. It’s Linux from top to bottom with a 2.4.19 kernel, KDE 3.03 and BIND 9. For the server trimmings it comes with up to date (as of January 2003) server programs like Apache, Samba, and NFS. SCO Linux 4.0 also comes with such mail essentials as Sendmail and Postfix and such developer necessities as gcc, cpp, and Tomcat.

If, however, you’re looking for a Linux with multiple Web server choices and every last new Linux program known to Freshmeat.net, you’re looking at the wrong distribution. SCO Linux 4 contains the most popular business Linux software choices and that’s about it.

In fact, if you know your Linuxes well and you look hard at UnitedLinux, you’ll find yourself thinking this look a lot like SuSE’s SuSE Linux Enterprise Server (SLES) 7.0. And, you know what? You’d be right. SLES 7 is UnitedLinux’s immediate ancestor.

What’s different about UnitedLinux isn’t so much the technology as the idea of providing business with a single common Linux server platform. With a common Linux platform, the UnitedLinux companies, and their major ally IBM, hope that independent software vendors (ISV)s take a permanent seat on the bandwagon. So far, it seems to be working. Borland, Computer Associates, NEC/Siemens, PeopleSoft, Progress Software, and SAP are all supporting it.

The Business of SCO Linux

In turn, this means, the four UnitedLinux companies-Conectiva, SCO, SuSE and Turbolinux–hope that Linux will move out of the popular, but low revenue, business of Web site hosting and file/print servers and into the much more profitable world of application and enterprise servers.

Of the four companies, only SCO is making a serious run at the North American reseller trade. Turbolinux is only a Far East play now. Conectiva… well, I know they want the Latin American market, but they seem to be making a hash of it.

Other than SCO, only SuSE is making a serious attempt at the business market, and that’s only in Europe. For all serious business purposes, SuSE is dead in North American market. That said, since they still have a US presence in the consumer space, you can still expect to find customers who want to consider it. Their lack of a viable reseller channel though means North American SCO partners won’t have much to worry about. European SCO vendors, though, are going to have their work cut out for them.

For us in the States and Canada, though, SCO Linux real competition is Windows 2000 and .NET Server, a point that SCO’s reviewer guide makes exceeding clear. SCO Linux also targets the major server Unixes-AIX, HP-UX and Solaris. The only Linux, it competes with is Red Hat’s Red Hat Advanced Server (RHAS).

SCO, having finally learned that OpenServer is here to stay, is wisely not targeting its own Unix market. As most of you already know, OpenServer is solid as a house and, it’s one of the safest operating systems out there according to the English security research house, mi2G (http://mi2g.com).

The Right Stuff, The Tech Stuff

To make SCO Linux 4 do its stuff, you’ll need at least an Intel 486, with 64MBs of RAM and 500 MBs of disk space to give it a try. But, that’s pointless. To do the jobs SCO Linux 4 is meant to do you’d need a minimum of a high-speed AMD Athlon or Intel Xeon with 512MBs of RAM and 40GBs of hard disk and up.

But, to really see UnitedLinux strut its stuff, with advanced features like IBM’s open source Memory eXpansion Technology (MXT) and large memory support so that even on Intel 32-bit architecture, UnitedLinux can address up to 64GBs of RAM with up to 4GBs per process, you need high-end servers with gigabytes of RAM and a Storage Area Network (SAN).

Since I don’t have one of those in the office (darn it!), I tested SCO Linux 4 on a HP Pavilion 512N with a 1.4Ghz AMD Athlon XP processor with 512MBs of RAM and an 80GB hard drive. By UnitedLinux standards, that’s barely getting into second gear.

Even so, some things quickly became apparent. One is that SCO Linux 4 is easy-I mean fall off a log easy-to set up. With more two decades of setting up server operating systems under my belt, I’ve never seen one this easy to set up before. In fact, I’ve found most desktop systems to be more difficult to install.

In large part that was because SCO’s Webmin and Usermin, Web-based administration programs are very easy to use. We also found, though, that YaST, the UnitedLinux default administration suite, also worked well.

For fine-tuning, though, they weren’t perfect. Both use KDE 3’s built-in Web browser, Konqueror 3.03, for their interface. And, I found that Konqueror consistently broke during some setup installations. For example, it always broke during some stages of setting up Samba, the Windows NT compatible file server. I was able to get around this by using Samba’s own SWAT administration tool. From some early experiments with Mozilla 1.01, the other supplied Web browser, it would appear that it works more reliable with the Webmin and Usermin administration tools.

While I didn’t test performance as such, I did run some informal tests of how fast it ran compared to Caldera’s pre-UnitedLinux Linux, OpenLinux 2.4. I found that on the exact same machine, SCO Linux 4 and its applications ran faster.

And, for lack of a better term, it ran smoother than its predecessor and other Linux distributions. There were fewer glitches. Yes, Linux is more stable than its competitors, but we all know programs that need fine-tuning before they work well enough for business. Well, on SCO Linux 4, there were simply fewer fit and polish problems.

That’s pretty amazing for a 1.0 release. Of course, if you look really closely, you can see a lot of bits and pieces still labeled SuSE rather than SCO or UnitedLinux, but for practical purposes of getting the job done, it runs remarkablly well for a 1.0 release.

Still, nothing is perfect and neither is SCO Linux 4. The biggest problem I found was that there is no graceful way to upgrade from OpenLinux 2.3. In talking with SCO, I discovered that it wasn’t just my own klutziness getting in the way. The only way to ‘upgrade’ OpenLinux, or any other Linux for that matter, is to back up your data and configuration files and restored them after letting SCO Linux 4 blow away the existing Linux system. If you’ve invested a lot of time in getting your Linux setup just so, be ready to re-do a lot of it.

So, in short, if you’re upgrading an existing business installation, make sure you have up to the second backups and allow for lots of time for bring the system back up to production level. Otherwise, you’re going to have one really ticked up client on your hands.

UnitedLinux representatives tell me that upgrade paths from older Linuxes will be made cleaner. But, with the possible exception of SLES users, I doubt that will happen. By its very nature UnitedLinux resets all those little, but vital, Linux file placements and settings to one standard way. And, that way, again with the exception of some of SuSE’s Linuxes, isn’t the UnitedLinux way. In the future, however, one SCO Linux 4 is in place upgrading SCO Linux Upgrade service, will go much easier.

How Much?

SCO, SuSE, and Turbolinux’s UnitedLinux-based distributions are available today. Connectiva is, in January 2003, lagging behind. For the North American reseller market, SCO took a quick and immediate lead and there are no signs that it will have significant competition from the other UnitedLinux companies in the near future. Europe, as I mentioned earlier, is another story. There, SuSE and SCO, as mentioned earlier, will compete head to head.

SCO’s pricing starts with a Base Edition, that’s meant for VARs or a small business with its own Linux expert, costs $599. Other versions, like the Classic Edition are $699, the Business Edition: $1,249 and the Enterprise Edition for $2,199. With each increase in price the owner gets higher levels of SCO direct maintenance and support with speed of response being the most important difference. The more you pay, the faster a SCO engineer will get back to the customer. At the bottom level, the reseller is responsible for all support.

All commercial versions also include the SCO Linux Update Service, which delivers upgrade and maintenance packs and security fixes. While each of the UnitedLinux partners has its own pricing system, you’ll find this basic tiered structure with better service for more money to be the same. For the full details see the SCO Linux 4 page.

Of course, you can also download UnitedLinux ISO images, but these come without support. And, as a business class operating system, technical support is the name of the game. If you’re an SCO reseller, I really wouldn’t worry about someone trying to steal your business without SCO’s support. They’ll be operating without a support network worth the name.

That said, it would be good to see a firm technical certification track set up for SCO Linux 4. Linux Professional Institute (LPI) president Evan Leibovitch has hinted that the LPI might brand their vendor-neutral certifications for UnitedLinux. Given, Caldera/SCO’s long support for the LPI certification, I expect the LPI certifications to become the de facto UnitedLinux/SCO Linux certifications.

What many resellers want to know though is whether SCO Linux 4, or any UnitedLinux, is better than RHAS. In my opinion, it depends on the customer. If you have someone who’s wedded to Oracle for their DBMS and Dell for their servers, chances are they’re going to like RHAS. But, for everyone else, but especially for IBM-oriented customers, I think SCO Linux is a good, viable choice.

ISV support, OEM support, technical support, and a strong, stable Linux distribution, there’s a lot to like here. With its pure business focus and pricing, UnitedLinux will never be the Linux for people at home, but SCO Linux 4, along with RHAS, is the Linux for people at the office.

November 6, 2002
by sjvn01
0 comments

Get some Backbone

It used to be choosing your Internet backbone provider was a serious, but not backbreaking decision. Things have changed. Now, the wrong choice could put you and your net-dependent customers-which is all of them these days-out of business.

It’s already happening. Last year, VARBusiness‘ readers picked WorldCom as their favorite backbone provider. Today, only a fool wouldn’t be looking for a way to get out of their WorldCom backbone contracts. Even, if you still believed that WorldCom divisions would keep working even as their leaders were shown to be crooks and the company continues its fall to bankruptcy, the wide-spread failures on October 2nd across the WorldCom backbone put some companies off the Internet and lead to wide-spread delays of two seconds and more for even customers. Heck, in just the days since I sent out the first version of this article to readers, a special report has damned Worldcom even more, the SEC is filing more charges, and as of this morning, they’re saying that they’ll have to restate nine (9) billion (BILLION!) dollars of income.

It’s not just the big names. Cable & Wireless, while well known in the UK, is quiet in the US even though it provides many US customers with their backbone connections thanks to buying out Digital Island in 2001, and via their purchase of Exodus, they’re also the company that hosts many American Web sites. What you probably don’t know is that C&W is very likely to be moving entirely out of the North American market. Some customers are already reporting that C&W has left them high and dry.

So it is that today, when you’re thinking about backbone services, you don’t want to just consider price, quality of service and support, you have to look closely at the backbone provider’s overall business health. These days, you need to read the financial pages as well as the technology section before picking a backbone partner.

Which backbone providers should you serve up to your customers? That’s a tricky question, considering all of the options out there. A good place to start is ISP Planet’s Backbone Directory. And these days you must follow up any companies that strike your fancy with a close look at their business news and financial reports. A useful place to start for beginners is Yahoo! Finance.

What to Look for in the Business Side

If you seen any these red flags, it’s time to give long, hard thought about your backbone provider. First, is the company laying off employees? Yes, in these bad times, everyone is doing some of that, but if they’ve gone through several layoff cycles, chances are they’re in deeper trouble than usual. Their stock may have gone up–stock analysts love companies that ‘cut out the fat.” But, the truth of the matter is that many tech. companies, ihncluding the backbone providers are now cutting muscle and bone, not fat. The market may love it, but what it means for you is that there will be fewer employees to keep your lines in order and gvie you support when they do go down. You can also be certain that those who are left will have poor morale and be overworked.

Are the officers of the company selling their stock? If it’s just one or two people, maybe they just want to buy a new house. If it’s three or more or the stock blocks are large, maybe though they’re getting ready to abandon ship.

Is the stock price below $5 a share? It is? Then the company has trouble. Unfortunately, that’s not uncommon these days. If, however, it’s below a dollar a share, then your backbone provider is in a fight for its life. Unless you have an outstanding relationship with the provider, you’ll probably want to find a new one. Has your company been delisted from the major exchanges? Yes? Then drop it. I don’t care if your mom does run it, it has almost no chance of surviving and when it goes down, it may take you with it.

What to Look for in the Technical Side

During the Net’s early days, anyone with a 1.544Mbps T1 or a 45Mbps T3 could call himself a backbone provider. If your customers don’t plan to run real-time programs, such as videoconferencing, traditional options like frame-relay over a T1 or T3 should work just fine.

But with the rise of broadband and multimedia applications, a backbone provider needs at least a 155.52Mbps OC-3 to be taken seriously. They also must have peering agreements with neighboring backbone providers so that Internet traffic can quickly reach its destination without any detours.
Clearly, not all T1s are created equal. Traffic congestion and network overhead ensure that no matter how fast a connection is supposed to go, it will never actually reach that speed. Many other factors–something as simple as what version of Cisco’s Internetworking Operating System you’re running on your router–can greatly impact effective throughput.

You also should check out a backbone provider’s basic infrastructure–and we’re not referring to copper or fiber optics. All too often, too much attention is paid to the network infrastructure of routers, switches and network connections, without any attention spent on the provider’s other basics.
For instance, a typical network operations center (NOC) used to have a half-dozen hard-core technicians living off Jolt Cola and cold pizza. But what your customers really need are backbone providers with several 24 x 7 NOCs. Each NOC should have automated trouble-ticket tracking and a top network-monitoring program, like Entuity’s Eye of the Storm, constantly running.

The real proof of the pudding, though, is how proactive the NOC is. With the best providers, the first sign that there’s a problem upstream should be when the NOC calls to let you know there’s a problem–not when your customers call you to scream about a network headache.

Another factor you should keep in mind is that some providers over-subscribe their backbones. You should ask providers exactly how many others will be sharing your section of backbone and then go with whoever gives you the lowest number if all things are equal.

Of course, savvy ISPs need more than one backbone provider to guarantee network availability. The best you can really expect from a backbone provider is 99.95 percent uptime. That’s a long way from the 99.9999 percent uptime that the harshest service-level agreements (SLA) require. The only way to achieve that type of availability is to use multiple providers.

Comparison Shop

If you don’t know your way around the market, choosing a backbone provider can be difficult. Fortunately, you can chew on Internet.com’s ISP-Lists mailing lists. These lists will give you a taste of what people really think about their backbone providers. The ISP-Bandwidth and ISP-Broadband lists are particularly useful for this.

Don’t have time for a list, but still have questions? Then simply search the list archives with the name of the providers you’re considering. Do keep in mind though that people tend to complain more than they praise though. You can always find someone who hates a given provider. The real red alert is when many people over time keep reporting the same problems with the same provider.

You also need to make sure that your provider can keep running even after disaster strikes. That’s a serious concern on the West Coast with major Internet exchanges like MAE West.

Not all disasters are natural ones. For example, you can expect California to see wave after wave of mandatory blackouts if the summer of 2003 proves to be another hot one. Even with the minimum serious power backups–diesel generators with automatic cutovers–you might want to consider including a backbone provider that doesn’t have many of its Internet eggs in California’s fragile basket.

Service Suggestion

Installation delays, poor technical support, and incompetent help-desk personnel can ignite heartburn. Again, the best way to avoid problems in the first place is to read ISP-specific mailing lists.
Ever been to a restaurant where the food was great, but you wanted to strangle the waiter because of poor service? Join the crowd. Backbone providers can be the same way.
Ken McLeod, CEO of CSG Wireless, puts it well. He looks for direct, person-to-person communication with his backbone providers. “Not an automated e-mail reply system, not a fax blast about pending orders, but a real human being who understands the network, can place and follow through on orders, and communicates delays. And, most important, I don’t want the dreaded call: ‘Hi, I’m Jim. I’m your new account rep this week!'”

Paying the Bill

You also need to look closely at your shopping bill. “From a business standpoint, we want short-term, one-year contracts because we need the ability to add or delete [providers] as necessary,” says BestWeb’s president Andrew Dickey. “Pricing needs to be competitive, but small differences do not matter. All of [our] suppliers have had reasonable contracts–but whenever the billing is complex, we have to check it carefully because I don’t think their accounting departments read the contracts the salespeople negotiate. Getting invoices corrected is difficult [and can turn into] horror stories.”

Indeed, you must keep a close eye on invoices. For example, some resellers have told me that even after C&W cut off their service, they were still getting billed for service. And, I myself have had trouble with DirecWay, which provides satellite Internet services. So, you should always have your accounts payable people go over your backbone bill with a fine-tooth comb.

Take a close look, too, at the provider’s Service Level Agreement (SLA) terms and conditions. What the salesman says and what the SLA says can be two different things. You must do your due diligence research before signing up with any backbone provider. Our point is, however, that you should strongly consider having some sort of independent auditing process in place to verify the performance levels that you’re paying for.

The plain truth is that you should be more concerned with business relationships instead of technology. A partner failing should be your greatest concern today not a hardware failure.

SIDEBAR: Align Your Backbone With the Right Bandwidth

Choosing the right bandwidth to connect to the backbone is a piece of cake–once you know which ingredients you’re looking for.
First, you’ll need to know whether your customers’ traffic is bursty, whether you have a high proportion of end users or you’re a portal (which determines how much traffic will stay in your network), and what type of applications or services you offer (real-time applications like videoconferencing or Voice over IP require special handling).
The options available to connect to the backbone are numerous, confusing and expensive, but we can try to simplify it. It’s important to understand the difference between speeds and technologies. T1 is a speed. OC is a speed. DSL, Frame Relay, ATM, Packet over Sonet and Ethernet are physical-layer technologies. These are often combined. For example, frame relay is often carved out from a T1, which rides on HDSL copper. If you’re not sure what you’re getting, ask and make sure you understand exactly what you’re getting in real world terms.
Perhaps the most common connection at the low end is the trusty T1 line (or a fractional T1, which is a provisioning of only some of a T1’s 24 64-bit channels). A T1 is just the speed, though–the technology is HDSL, the daddy to the widely available ADSL, SDSL, etc. T1 and its big brother, T3 (or DS3), generally run over copper wire.
Also on the low end is frame relay, a shared, packet-switching protocol, which is great for inexpensively connecting branch offices together over long distances, but it’s not really suited for a service provider to connect to the backbone because of the inefficiencies in encapsulating IP within frame relay.
Many users would never notice the difference, but if you’re running a real-time application like a VPN or videoconferencing, you’ll see the inferior quality of service in a hurry.
Stepping up to even higher bandwidth levels comes ATM, with a maximum limit of 622Mbps. Basically, if you have ATM, you should continue with it, but it’s complicated and expensive.
Fiber has become widely available (thus dropping its price), which makes Packet over Sonet (whose speed is defined by Optical Carrier) a more affordable backbone connector. Sonet (synchronous optical network) defines rate and interface connections over a fiber-optic line.
Fiber loops are indeed loops. THis means that if a line is cut, the traffic is automatically routed along another path. It also has fewer problems with interference, making it an excellent transport medium over long distances

Unfortunately, the bottom has fallen out of the Fiber market. Global Crossing, once a name that meant the future of the Internet, now struggles to survive. Still, you can get great deals on Fiber. Cogent Communications, now owners of venerable PSINet, for example, sells 100Mbps OC48 connections for a jaw-dropping $1000 a month. Of course, it’s only available in currently 21 metro areas, but still it’s amazing.

While Cogent can do some of this by using a simplified optical system that doesn’t use pricey ATM, you have to tread carefully anytime something that sounds to good to be true. The history of Internet connectivity is littered with ISPs and Competitive Local Exchange Carriers (CLEC)s that had connectivity offers that were too cheap to be true-or more to the point, profitable enough to keep them in business and you in Internet connectivity.

Last, but not least, is Gigabit and 10-Gigabit Ethernet. While not widely available yet, Ethernet is running on a majority of LANs, which makes it easy to connect to an Ethernet interface on the backbone. The problem is that there’s a distance limitation to Ethernet, which usually requires co-location of the routers and switches. Still, I think that these technologies will eventually play a major role in metro area network backbones. Considering the economy, though, eventually might well be 2004 or 5.

Rev the Net

Bandwidth Designation
155.52Mbps OC-3
466.56Mbps OC-9
622.08Mbps OC-12
933.12Mbps OC-18
1.244Gpbs OC-24
1.866Gbps OC-36
2.488Gbps OC-48
4.976Gbps OC-96
10Gbps OC-192
13.21Gbps OC-255

November 4, 2002
by sjvn01
0 comments

Linux Certification: What’s The Value?

By my count, there are four Linux certification programs. These are Sair Linux and GNU Certification’s Linux Certified Professional(LCP); Linux Professional Institute (LPI); Red Hat Certified Engineer (RHCE); and CompTIA’s Linux+. It’s too bad that so few jobs require Linux certification.

Oh, there are Linux jobs out there; it’s just that they’re much more likely to call for a bachelors in computer science than a Linux certification. In my analysis of 500 Linux job postings on Monster.com, the Washington Post job section, HotLinuxJobs, and HotJobs, only 1% of Linux jobs asked for any certification and when they did the most popular was the RHCE.

Instead of certifications, what employers want are people with four year computer science degrees and three years and up experience in Linux.

What are certifications for? Well, according to Evan Leibovitch, president of the Linux Professional Institute, in a recent article in Certification Magazine, have a “single specific purpose. They are intended to separate the people who ‘know their stuff’ from those who don’t.”

Most employers think of all certifications, with the exception of Cisco’s, as being little more than minimal standards. The most common attitude seems to be, “We still have to train them no matter how many pieces of paper they have.”

The LPI also takes a strictly vendor-neutral approach to their certification efforts. The other non-Red Hat certifications also take this approach, but the LPI takes this much more seriously.

Historically, though, that’s not been the most successful route. The important certifications in the job market, like the Microsoft Certified Software Engineer (MCSE), Certified Novell Engineer (CNE) and the Cisco Certified Internetwork Expert (CCIE), have been vendor driven. Indeed, one major reason these certifications have become so popular is that partnership relationships with each company often required that a reseller or integrator have a certain number of certified staffers on board. Given this, it should come as no surprise that what interest employers do show so far in Linux certification is in the RHCE.

Another problem seems to be that while those in the Linux community tend to know about Linux certification, potential employers don’t know them. In a recent CRN survey solution providers are clearly getting the idea that Linux is the next coming thing, but they still see certification as a “key sticking point.” And, they also think that, despite the track record, vendor-neutral certification is still the way to go.

With all this, should you still bother to get certified? If you plan to be employed in Linux in the long run, it’s probably worth it. Some companies, like IBM are strongly encouraging their partners to get either LPIs or RHCEs. Indeed, IBM will pay its middleware partners up to $3,000 for their Linux certification costs. With companies like Dell, HP, and Oracle also lining up behind Linux, it’s likely that they too will eventually support certification and with this support, Linux certification, in turn, will become important for job-seekers.

Walking The Linux Walk

In the short run, for both potential employees and employers, a Linux certification shows that someone is serious about Linux. Lots of people can talk the Linux talk, but can barely walk around a KDE desktop. With a Linux certification, you can at least be sure that the holder is trying to master Linux rather than play with it.

Another current reason to get a RHCE is that, according to a December 2001 salary survey by Certification Magazine is that at shops that do recognize this certification, RHCE holders make 9.6% more salary per year. Now if only more employers paid attention to Linux certifications, there’d be no question about the need to get one.

# CompTIA’s Linux+ — CompTIA is best known for its technician level certifications like A+, for hardware, and the self-explanatory Network+, Linux+ is another entry-level technician certification. To get a Linux+ certification you must pass a single two-hour test. More than the other certifications, the emphasis is on hardware and system maintenance. In short, it’s exactly what you think it would be: a certification for a technician, not an administrator or developer.

Given the popularity of CompTIA’s certifications, the Linux+ would work well for someone who likes getting their hands dirty with the nitty-gritty of hardware, while wanting to add some Linux expertise to their resume.
# LPI – The oldest and most broadly supported Linux certification is supported by almost anyone who’s anyone in Linux-SCO (fomerly Caldera), IBM, SGI, SuSE and VA Linux-except for Red Hat. This three-level certification program is meant for serious Linux administrators. For example, someone with a Linux+ could be expected to know the basic use of sed, someone with an LPI level 1 could be expected to know how to write basic shell scripts with sed.

Eventually, LPI will be an important certification for anyone looking for a Linux job. For now, it’s the certification to get if your job hope is to work with a Linux company or an extremely Linux savvy business, like say an IBM partner using Linux, that’s not wedded to Red Hat.
# RHCE – If an employer knows any Linux certification, it’s this one. That’s no great surprise; Red Hat is following the tried and true route of bundling their certification with what’s easily the most popular business Linux distribution in North America. And, in all fairness, it is a good set of certifications. Fairfield Research conducted an industry-wide survey 3,939 certificated IT professionals in late 2001, and RHCE, the one Linux certification to appear, got top marks.

An RHCE is meant to say that the holder is a qualified Red Hat Linux system or network administrator. Given our analysis of job ads, employers agree. With Dell and Oracle’s support, the RHCE should continue to be an important certification.
# Sair Linux and GNU Certification – The LCP is another entry-level certification that sits between the Linux+ and LPI’s Level One. Like the LPI, it’s vendor neutral.

While Sair has a good reputation, it’s hard to see the LCP becoming an important certification. The LPI and RHCE have the industry connections and CompTIA’s certifications are well-known in all kinds of business. During the last week, when you tried to click on Sair’s Why Certify page, you would have found a 404-error message-page not found. What more need be said?

So what should you do? If you’re looking for a Linux job, there really isn’t a pressing reason to get a Linux certification. A RHCE is the most likely to help you, and eventually an LPI or Linux+ could give you a leg up. But, for now, stay in college and get some kind of job where you can work with Linux, a degree and experience will serve you much better than a certification.

A version of this story was first published in Linux Planet.