July 10, 2018
by sjvn01
0 comments

Continuous integration and delivery tool basics

CI/CD tools are key to today’s agile, container-driven software production cycle. This “explain like I’m 5” overview helps you get started.

Once upon a time, back when waterfall was the primary software development methodology, you could spend months or years working on a single software project. With the help of agile methods, programming cycles were reduced to days or weeks.

How quaint all that looks now! Weeks? Ha! In today’s DevOps-driven world, we use continuous integration, continuous delivery, and continuous deployment (CI/CD) to update programs in time frames measured in days or hours. For example, with its CI/CD pipeline, Netflix takes less than half an hour to move its programs from code check-in to multi-region deployment.

Can you say fast? I knew you could!

Automation is a key part of the CI/CD process, so naturally there are several tools that aim to help you manage the tasks. To set the stage, let’s review the basics of CI/CD before I give you an overview of some popular CI/CD tools.

How CI/CD works: A short refresher

In CI, developers integrate their code into a shared code repository’s main branch early and often. The goal is to reduce development costs by dealing with issues as soon as possible.

With continuous integration, the software is not written in isolation and then merged into the larger system at the end of a development cycle. Typically, code is integrated multiple times a day. Thus, programmers can resolve code conflicts and other issues while it’s still relatively easy to fix them.

Doing this manually would be incredibly time-consuming and annoying. For practical purposes, nearly every CI setup uses automated test suites. When code is ready to merge, an automated process kicks off a new codebase build. The automation also runs a suite of quality assurance tests to check for bugs and determines if the update introduces integration problems.

Continuous delivery is the next step after CI, in which you also automate release cycles. This may mean deploying new application versions when the software passes automatic tests. For instance, one such test, called a canary release, rolls the application out to a small subset of users; after getting their sign off, the system releases the software to production.

Finally, once an IT shop is comfortable with continuous integration and delivery, it can attempt continuous deployment. New code is continuously released to users in a process called a pipeline. All changes that pass successfully through the production pipeline’s automated tests are automatically released. That means the end of release days or Patch Tuesdays. When all works well, the code moves smoothly and automatically from programming to testing to release.

To make all this happen—and achieve the “when all works well”—you need the right tools. Here are some of the best known ones.

Jenkins

If you know of any CI tool, chances are it’s Jenkins. This open source program, which is supported by CloudBees, is one of the oldest and best-regarded CI programs. It also has about 2 million users, making it the most widely deployed CI program.

Jenkins is a cross-platform CI tool that can be configured both through a GUI interface and console commands. Jenkins’ CI process can be defined either declaratively or imperatively using the Groovy language or with text boxes using Jenkins’ web user interface.

Jenkins is noted for its flexibility. In large part, that’s thanks to the ease of adding plugins and the power provided in doing so. This can be a mixed blessing. Since everything is done using plugins, from listening for web hooks and watching repositories to supporting Docker, some DevOps consider Jenkins to be fragile.

Written in Java, Jenkins began as a fork of Sun/Oracle’s Hudson in 2011 and was released under an MIT license.

f you don’t want to run Jenkins on your own servers, CloudBees offers a hosted solution. CloudBees also has a full analytics platform, but you have additional options; advanced analytics and dashboards also are available from Jenkins’ large open source community.

With more than 1,400 plugins, Jenkins’ flexibility and broad support demands you give it a try. It’s especially popular with GitHub build integration, but it can be used for almost any conceivable CI/CD job.

Buildbot

Buildbot is a popular CI framework with a long history. It dates all the way back to 2003, when Mozilla began using it. While Mozilla depreciated Buildbot in favor of Taskcluster, the older tool is still used by other well-known open source projects such as Chromium, GNOME, and WebKit. Buildbot itself is written with the Twisted Python library and licensed under the GPLv2.

A Buildbot installation comprises one or more controllers and multiple workers. The controllers monitor source-code repositories for changes, coordinate workers’ activities, and report results to users and developers. Workers run on all major operating systems and version control systems.

To configure Buildbot, you provide a Python configuration script to the controller. These scripts can be simple, using built-in components, or expanded with Python. That makes it an easy choice for any pythonista, and suggests a steeper learning curve if you don’t know Python.

At heart, Buildbot is a job scheduling system. It’s used primarily to automate build testing across a wide array of platforms. If that’s all you need, check it out. Need more? Keep looking.

CircleCI

The CircleCI tool is admired for its speed. That’s because CircleCI caches builds and can run tests in parallel over multiple machines. The net result is quick test times. It’s also appreciated because it can be run in the cloud or as an on-premises version.

You can use CircleCI on almost any operating system or cloud. CircleCI is a single-page web application that makes heavy use of AJAX and HTML5. What you can’t do with it is build Windows applications. While you can build applications using .NET Core under Docker with CircleCI, that’s far from full Windows building or test support.

The program gets good marks for usability. Explains Forrester in a report, its “setup wizard, with its out-of-the-gate YAML Ain’t Markup Language (YAML) examples and next-step guides, is one of the most developer-friendly tools for configuring projects. Reference customers were impressed with the simplicity and flexibility of setting up build notifications and alerts.” Among the other advantages cited are its build diagnostic capability via Secure Shell (SSH), workflows, and Docker support.

CircleCI build configurations are stored in a file called circle.yml, which lives in each project’s root directory. By treating CI configurations like any other source code, it’s easy to back up and share CircleCI configurations.

In addition, CircleCI works in conjunction with other common developer tools, such as Slack, HipChat, and IRC, as well as testing tools like Code Climate and Sauce Labs. There’s also an option to notify you only if the build status changes.

While it is not an open source program, the company says you can build open source projects on CircleCI.

If you’re wedded to open source or Windows, you can skip CircleCI. But if that’s not you and you need really fast testing, CircleCI deserves your attention.

Spinnaker

Netflix knows a thing or two about CI/CD. Spinnaker, its continuous deployment tool, is a microservices application based on a set of Docker containers. Since 2014, every time you’ve enjoyed watching “Stranger Things” or “The Unbreakable Kimmy Schmidt,” you can thank Spinnaker for their smooth delivery to your television.

Spinnaker can be deployed in Docker Swarm or Kubernetes clusters and be set up in Amazon Web Services (AWS), Azure, and Google Cloud using templates. Spinnaker can package application artifacts, such as binaries and container images. You can also use Spinnaker with other CI tools, such as Jenkins and CircleCI, which may be a good option for a shop that doesn’t want to standardize on a single tool. For all its good points, you should know Spinnaker is not a build tool; it’s a testing and deployment tool. The build part is handled by Jenkins or other CI tools.

For CD, though, Spinnaker demands your attention. Spinnaker boasts a multicloud approach and pluggable architecture. Its predecessor, Asgard, had been locked in on AWS.

Google joined forces with Netflix with shared goals in creating open source Spinnaker. As Google product manager Christopher Sanson explains, “Out of the box, Spinnaker supports sophisticated deployment strategies like release canaries, multiple staging environments, red/black (a.k.a. blue/green) deployments, traffic splitting, and easy rollbacks.”

This makes Spinnaker especially good at testing and deploying applications across entire server fleets very quickly. Thus, Netflix uses it to constantly update its software at a breathtaking rate.

Among its virtues is its wide support on nearly every platform. Spinnaker currently runs on AWS EC2, Kubernetes, Google Compute Engine, Google Kubernetes Engine, Google App Engine, Microsoft Azure, and OpenStack, with support for Oracle Bare Metal and DC/OS (Datacenter Operating System) coming soon. You can also work with Spinnaker’s open source code, which is under the Apache 2 license.

Zuul

Zuul is an open source CI/CD program that’s been hiding in plain sight since 2012. You probably overlooked it because its creators were using Zuul as the in-house CI/CD system for the then open source OpenStack cloud program. OpenStack started its CI work with Jenkins but decided it needed something more to handle its complex, multiple-project needs.

OpenStack, if you don’t know it, is made up of over 50 open source programs. Managing its numerous components from multiple contributors was, and is, a challenging job.

The Zuul project’s focus is on multi-repo gating. In this methodology, Zuul strives to automatically ensure correctly ordered testing of changes in parallel, which in English means making sure things happen in the right order. This speeds up the CI process so developers don’t have to wait forever for their changes to make it into the code repository.

Zuul accomplishes this with its dependent pipeline manager, which enables parallel execution of test jobs for gating while ensuring changes are tested correctly, exactly as if they were tested one at a time. How? By performing speculative execution of test jobs. Zuul assumes that all jobs will succeed and then tests them in parallel. If they do succeed, all the code can be merged, which certainly speeds up software integration.

However, if one code snippet fails, the batch is reexamined; changes that were expecting it to succeed are re-tested without the failed change. In the worst case, changes are tested one at a time. This makes Zuul ideal for handling large, complex projects.

The OpenStack Foundation recently spun out Zuul from the main OpenStack project. Zuul is open source and licensed under Apache 2. If you’re dealing with complex projects, spend some time investigating its features.

The future of CI/CD

This is far from an exhaustive list. There are dozens of others.

Before deciding which tool is right for you, look at your projects and what you really need to move them into the 21st century. For example, Zuul is overkill for most projects. Others, such as Spinnaker, are great for testing and delivery, but they’re not meant for integration.

Before looking at the tools, decide on which functionality you need. There’s no point in investing in Jenkins, which is great at CI, when what you really want is a CD tool for your existing CI system.
Do you want to self-host or use a cloud-based service? What are your company’s data storage rules? Which fits better in your budget? Keep in mind if you self-host, you need an in-house DevOps team to run it.
What do others in your field think about these services? Is there already a recognized best-of-breed CI/CD tool in your business? Or, as with Kubernetes, must you face the sad fact that there is “no single best set of tools” for CI/CD. In any case, you need to do a lot of testing to find the right fit for your company.

Best practices in CI/CD are still a work in progress, and so are the tools that support them. It’s likely that we soon will see a shake-up as the best CI/CD tools prove themselves in the marketplace and on DevOps’ desktops.

While Netflix, CI/CD’s poster child, can complete integration, testing, and delivery in hours, most companies are far from reaching that level of efficiency. Given the business advantages, it’s a goal well worth working toward.

CI/CD is not an end to itself. But when you couple it with your business goals, you’ll be taking a major step forward in improving not just your software deliverable times, but your bottom line as well.

CI/CD tools: Lessons for leaders

There are many significant CI/CD programs. For now, there are no true market leaders. Expect to try out several to find the one that best fits your business needs.
Before you decide on a CI/CD tool, analyze your corporate CI/CD needs. This means you must work closely with your development team.

Continuous integration and delivery tool basics was first published in HPE enterprise.nxt

May 30, 2018
by sjvn01
0 comments

Back to basics: What sysadmins must know about logging and monitoring

With containers springing up and down in minutes and virtual machines coming and going in hours, some sysadmins have neglected their system logs. Log files still provide invaluable insight into how systems are operating! Hereâ s what you — still — need to know.

When I was a young whippersnapper, we had to work our server logs 29 hours a day for tuppence a month, and the CIO would beat us around the head and neck with a broken bottle, if we were lucky! But, sheesh, you tell that to young sysadmins today, and they won’t believe you!

With the rise of containers and virtual machines, some system administrators have been neglecting their system logs. That’s a mistake.

Even if your containerized applications spin up and down several times an hour, you still need to keep and analyze logs. To find the root cause of a failure or to track down a system attack, you must be able to review what happened, when it happened, and what components of your software and hardware stack were affected. Otherwise, you’ll waste time looking for problems in the wrong place — time that you don’t have to spare in an emergency. Or, worse still, you may miss hidden issues such as performance problems, security violations, or costly use of system resources.

Without system logs, you’re not administering a system; you’re herding data into a box and hoping for the best. That’s no way to run servers, whether they are physical, virtual, or containerized.

So, here are some of the basics to keep in mind as you approach server logging in the 21st century. These are all practices that I either use myself or picked up from other sysadmins, including many from the invaluable Reddit/sysadmin group.

Centralize your logs

First, decide where to keep the log files. While there’s still a time and place for keeping logs in /var/logs, in the case of Linux/Unix, or in %WINDIR%\Panther, in the case of Windows, most sysadmins and auditors prefer a centralized logging system. After all, who wants to search for a problem over hundreds of log files on as many servers? In addition to simplifying your log analysis, you can more easily secure log data if it’s in one location.

Also, centralized logs make it easier to conform with regulatory compliance mandates imposed under HIPAA, SOX, GDPR, etc.

There are numerous centralized logging system programs, such as Splunk, Fluentd, and Graylog. Based on the sheer number of mentions I’ve seen from sysadmins, however, the plurality build their own management tools from the ELK stack, a.k.a. the Elastic Stack.

ELK is made up of three open-source projects: Elasticsearch, Logstash, and Kibana. Elasticsearch is a search and analytics engine. Logstash is a serverâ side data processing pipeline that imports data from multiple sources simultaneously, transforms it, and then sends it to a “stash” like Elasticsearch. Kibana enables its users to visualize data in Elasticsearch. This approach enables IT departments (and their admins) to create a custom logging and monitoring mix that fits their company’s precise needs.

How long should you keep log data?

Once you start centrally collecting your data, the next eternal question is “How long should you keep your log data?” The answer is: “It depends.”

Don’t just rely on your gut feeling to make this call. You need to consider the legal requirements for retention or destruction, your company’s own data retention policies, and how long the logs may be useful.

How much room they take up is also an issue, but in today’s world of cheap storage, it should be the last one on your list. In addition, big data log analysis tools such as VMware’s vRealize Log Insight and programs based on the SMACK stack can wring useful analysis from even the largest collection of log files.

That said, when you go metal detecting on a beach, you keep the coins; you don’t keep all the sand as well.

Monitoring basics

Once you have all that log data securely snuggled away, you need programs to help you monitor the data (and turn it into information). There are a multitude of choices, but before you choose one, you need to define what your organization needs.

Consider questions such as: How many servers? Using which network gear? Cloud? Containers? More than one data center? What configuration management tools are you using? What’s your budget? The list goes on and on. But, until you know the answers, you’re not ready to commit to a monitoring tool.

Generally speaking, as one high-level sysadmin at a major retailer recommends:

Anything backed by a relational database does not scale adequately. Choose a time series database whenever possible.
You must ensure high availability.
A single tool that does two things adequately is better than two platforms, each of which does only one thing really well. In the real world, you don’t want granular data (e.g., “One of your switches has rejected 79 percent of packets on interface 401-ew-13-gw-west”). You’d rather know that a switch is down, which explains why 12 application servers can’t talk to your back end.
Data granularity matters. “If you only poll data every 3 to 5 minutes, there’s a lot that can happen during that time that you will never catch.”

The sysadmin’s specific recommendations: “For a shop running 100 machines in an office? Sure, Zabbix is probably fine. You’re 100% Windows? Great, use SCOM. You only care about network equipment? PRTG or Solarwinds is probably fine. Web performance is all you care about? Site24x7/Pingdom are great.”

Many others agree with these recommendations. Other programs that people in the field are pleased about include DataDog, Prometheus, and LogicMonitor.

ou may have noticed I’ve not mentioned older-style system monitoring programs such as Nagios. It’s not that Nagios is no longer useful or being used. But Nagios was designed before companies were deploying across multiple data centers or the cloud. It doesn’t scale well across today’s platforms.

Nagios is also check-based. That is, the software checks: “Is the CPU over 70%? Then the check has failed; take some action.” Generally, when a check fails, Nagios sends an alert. Prometheus, a newer program, monitors a metric over time; if the metric doesn’t meet a configurable threshold or baseline, the application sends an alert.

Both approaches have their virtues, however. There’s still lots of life left in Nagios.

When it comes to logging and monitoring tools, don’t search for a silver bullet. There is none. Each tool has advantages and disadvantages to weigh against your previously itemized company needs.

Before you buy or set up any software, be wary of programs that promise unlimited logging with little effort and no need for agents or syslogs. Also be careful of programs that offer “value-added functionality,” such as automatically shutting down a rogue device, which may cause more trouble than it’s worth.

Tips and tricks of the logging trade

Another important point with log keeping—and one I feel almost embarrassed to mention—is that you must use the Network Time Protocol set to Zulu time, a.k.a. Greenwich Mean Time, on your servers. This way your logs have the correct time and can be correlated correctly with one another, no matter from where the data is gathered. Without the use of Zulu time, you can’t use timestamps to correlate between events.

You might think that this is a “duh” point to mention, but I recently encountered an article suggesting that users set their Amazon Web Services to local time. No!

Set alert thresholds appropriately from the start. If you don’t, you will get awakened in the middle of the night by a default-level alarm that you don’t really care about. That’s not a threat. It’s a promise.

This is important. Alert burnout is real, and it can cause you to begin ignoring alarms, which eventually will be catastrophic. If the information you’re monitoring isn’t going to change what you are doing, you’re wasting your time. Alerts need to be actionable. Otherwise they’re just useless noise.

My rule of thumb is simple: Alarms should be alarming. Everything that comes to my phone should make me shoot out of bed to fix the problem. Everything else should be relegated to email (subject to various inbox rules) or, if really low-priority, a summary report. If you set your phone alerts so that you get 10 messages a day, you will stop checking your phone after two days.

You’re better off monitoring a small number of carefully chosen things that matter than a large number of things that don’t. For example, people often monitor network utilization on all servers as their first step, rather than checking on the most critical services. The result? Stuff is still down, and they have huge volumes of useless data collected. Understand where your critical problems are most likely to happen, and check those servers first.

You should also use your captured data for more than just troubleshooting. For example, use it to predict trends and future hardware investment.

The usefulness will become clear when it comes time to negotiate about departmental budgets. Management will listen better if you say, “Over the past year our storage-area network disk usage has increased by 5TB. Based on that, I expect to need another 5TB for the coming year, so we will need additional storage to accommodate that growth.” That’s more credible than, “I think we need more storage this year.”

Get the picture? Logging and monitoring remain just as important as ever. You ignore them at your risk and your company’s peril.

Logging: Lessons for leaders

Logging is as important now as it has ever been.
Logging remains vital for detecting technical, security, and cost problems.
Logging is also useful for predicting capital and service expenses.

Back to basics: What sysadmins must know about logging and monitoring was first published in HPE

January 11, 2018
by sjvn01
0 comments

The real history of the modern Internet

he transition from the Internet’s 1960s ancestor, ARPANET, to our Internet changed everything. Two developments made this happen. The first (and more obscure) event was the creation of the Commercial Internet Exchange (CIX). The second change, which almost everyone knows, is the creation of the World Wide Web.

In 1989, you couldn’t sell a cat on the Internet, never mind setting up an e-commerce site to sell books. Commercial enterprises were expressly forbidden from participating in any Internet activity.

Not that the activity was easy. If you wanted to work on the ‘net, you used character-based interfaces with arcane syntaxes such as Telnet, FTP, Archie, and Gopher. An easy-to-use hypertext, graphical way to navigate was still years away. If you wanted to hang out, chat, or work online, you used modem-driven online services such as America Online (AOL), CompuServe, or GEnie.

CIX and the single network

In the late 1980s, many TCP/IP networks joined together to form what we now refer to as “the Internet.” Besides ARPANET, these groups included the U.S. Department of Defense’s MILNET, academia’s CSNET, and the National Science Foundation’s (NSF) research and educational network, NSFNET. At that point, the Internet was not for ordinary people or businesses. As users moved from the academic Internet into the “real” world—such as graduating from college and accepting their first jobs—they wanted Internet access.

That began to occur with the rise of commercial email systems. In 1989, the Corporation for National Research Initiatives (CNRI) developed an email gateway between NSFNET and the most popular business email system, MCI Mail. The idea of emailing someone across computer networks exploded. (Initially, someone in Columbus, Ohio, drove magnetic tapes to and from CompuServe’s offices and a nearby university, taking “sneakernet” to an illogical extreme.) Soon, everyone was trying to connect their email systems to the Internet. They used both the eventual standard, the RFC 822 user_name@domain.top-level-domain format we all use now, and a wild mix of other standards, such as UUNET’s Bang addressing and X.400. But email was relatively simple.

More important, nobody could charge money for Internet access or allow any kind of commercial activities. The early Internet services dealt with the government, research institutions, or schools, not with individuals. The Internet’s acceptable use policies forbade for-profit activities and “extensive use for private or personal business.”

IBM, Merit Network, and MCI provided business services over national and regional networks. To save money and expand their reach, in 1990 they formed the nonprofit Advanced Network Services (ANS), which created the first commercial Internet backbone, ANSNET. This same wide-area network was also used by NSFNET, which needed more bandwidth. For the first time, that meant the corporate network and the older Internet were running on the same cables and routers.

ANSNET did more than increase the early Internet’s backbone speed from T1’s blazingly fast 1.544 megabits per second to T3’s then-amazing 44.736 Mbps. In 1993, the NSF also agreed to let the trio of pioneering Internet companies form ANS CO+RE Systems, a for-profit corporation that sold corporate Internet access—as long as they didn’t use it for advertising. Little did they know what was to come!

This agreement opened Pandora’s box. Now, everyone wanted to know how to pay for the rapidly expanding network traffic, as data jumped from business to nonprofit networks and back again. To hash out these issues, a public mailing list, com-priv, was set up on an early Internet service provider, PSI Network (PSINet). From these conversations, three ISPs that were not part of ANSNET (CERFNET, PSINet, and UUNET), formed their own network of networks: the Commercial Internet Exchange (CIX).

Physically, at first, CIX consisted of a single router that connected their three networks. Doesn’t sound very grand, does it? Yet, according to Frank Dzubeck, president of Communications Network Architects, a Washington, D.C., consulting firm, this was the creation of the “backbone of the Internet.”

That’s because so many other ISPs joined them. In no small part, this was because all the CIX companies agreed to charge each other a flat fee for sharing their network traffic, instead of charging a fee based on how much data was carried.

That was the real beginning of net neutrality, although Tim Wu wouldn’t coin the phrase until 2003.

However, one ISP, ANS, wouldn’t agree to peer—that is, to check traffic with CIX in a net revenue-neutral way. This made it impossible for some groups on one “side” of the Internet to connect with the others.

This troublesome situation continued until 1992, when Mitch Kapor, founder of Lotus Development and its killer-app Lotus 1-2-3 spreadsheet, became chairman of CIX. Kapor got the two sides to agree to a “great compromise.” In the agreement, ANS and CIX agreed to share traffic across each other’s networks. As Kapor said at the time, “In taking this significant step, we enable greater freedom from content restrictions on the Internet.”

With CIX, commerce immediately started flooding the web. And it won’t surprise you to know that spam was there from the start. While unsolicited commercial messages began even earlier, in 1994 a pair of Phoenix attorneys launched the first major commercial spam campaign about a Green Card Lottery.

It wasn’t until the web came along that e-commerce really started to gain traction. CIX laid the foundation for both great failures, like Pets.com, and great successes, such as Amazon.com.

CIX transformed the collective networks into the Internet you know today: a network in which you can go to any website in the world without worrying about the underlying network.

The spinnerets of the web

That’s a good thing because while ANS and CIX were working out the network and business kinks to unite the Internet and open it to anyone, Tim Berners-Lee came up with the idea of the web, an Internet-based hypertext system.

It wasn’t a new idea. Credit goes to Vannevar Bush in his July 1945 article, “As We May Think.” Ted Nelson’s 1960 Xanadu hypertext vision also pointed toward the web. And, ironically, Apple could have been much bigger than it already is had it simply made the Apple HyperCard network-aware.

Be that as it may, Berners-Lee is the individual who took the hypertext idea and turned it into the web reality. He did it not to share LOLCats images (however good an idea that might have been), but to help researchers share ideas at CERN, the European Particle Physics Laboratory.

Berners-Lee used NeXT, the BSD Unix-based computers that are the modern-day Mac’s most direct ancestor, to create the first web server: info.cern.ch. With the help of Nicola Pellow, a visiting graduate student who created the first web browser, the World Wide Web was off to an inauspicious start in 1991 with the “publishing” of the CERN telephone directory.

It did not make a great first impression. Adoption was slow. It wasn’t until 1993 that I wrote the first mass-market review of the WEB.

The web would have taken off without my help. With CIX and ANS making peace, ISPs sprang up throughout the world, offering Internet access at the unheard speed of 28.8 kilobits per second. The only thing needed now was access to an easy-to-use program that would let users search and play and work with what they found on the Internet.

Enter the graphic web browser.

The first popular graphical web browser was from the National Center for Supercomputing Applications at the University of Illinois at Urbana-Champaign. Mosaic, created by Marc Andreessen and Eric Bina, wasn’t the first graphical web browser. ViolaWWW, a Unix browser, takes that honor, and Cello was the first Windows graphical web browser. What Mosaic had going for it was it let you see images within pages. Earlier browsers could only show images as separate files. It was no contest: Mosaic would dominate the first browser war.

It wasn’t easy for most people to use the Internet. Windows, the most popular desktop operating system, didn’t natively support TCP/IP until Windows 95 appeared. If you wanted TCP/IP on Windows before then, you had to use the notoriously difficult Trumpet Winsock program. (OS/2 had its own web browser built in, which at the time was a real bragging point.)

Still, people loved the idea of the Internet. But none of that—not Facebook, Twitter, Internet memes, Netflix, or World of Warcraft—would exist if it hadn’t been for CIX and the web.

CIX made the Internet open, available, and affordable for anyone. The web made the Internet’s resources so easy to use that my 2-year-old grandson can play with it.

June 22, 2017
by sjvn01
0 comments

The What, Why and Wow! Behind the CoreOS Container Linux

The usual debate over server Linux distributions begins with:

Do you use a Red Hat Enterprise Linux (RHEL)-based distribution, such as CentOS or Fedora; a Debian-based Linux like Ubuntu; or SUSE?

But now, CoreOS Container Linux joins the fracas. CoreOS, recently offered by Linode on its servers, takes an entirely different approach than its more conventional, elder siblings.

So, you may be asking yourself: “Why should I bother, when there are so many other solid Linux distros?” Well, I’ll let Greg Kroah-Hartman, the kernel maintainer for the Linux-stable branch and CoreOS advisor, start the conversation:

(CoreOS) handles distro updates (based on the ChromeOS code) combined with Docker and potentially checkpoint/restore, (which) means that you might be able to update the distro under your application without stopping/starting the process/container. I’ve seen it happen in testing, and it’s scary [good].”
And that assessment came when CoreOS was in alpha. Back then, CoreOS was being developed in?—?believe it or not?—?a Silicon Valley garage. While CoreOS is no Apple or HPE, it’s grown considerably in the last four years.

When I checked in on them at 2017’s CoreOS Fest in San Francisco, CoreOS had support from Google Cloud, IBM, Amazon Web Services, and Microsoft. The project itself now has over a thousand contributors. They think they’re on to something good, and I agree.

Why? Because, CoreOS is a lightweight Linux designed from the get-go for running containers. It started as a Docker platform, but over time CoreOS has taken its own path to containers. It now supports both its own take on containers, rkt (pronounced rocket), and Docker.

Unlike most Linux distributions, CoreOS doesn’t have a package manager. Instead it takes a page from Google’s ChromeOS and automates software updates to ensure better security and reliability of machines and containers running on clusters. Both operating system updates and security patches are regularly pushed to CoreOS Container Linux machines without sysadmin intervention.

You control how often patches are pushed using CoreUpdate, with its web-based interface. This enables you to control when your machines update, and how quickly an update is rolled out across your cluster.

Specifically, CoreOS does this with the the distributed configuration service etcd. This is an open-source, distributed key value store based on YAML. Etcd provides shared configuration and service discovery for Container Linux clusters.

This service runs on each machine in a cluster. When one server goes down, say to update, it handles the leader election so that the overall Linux system and containerized applications keep running as each server is updated.

To handle cluster management, CoreOS used to use fleet. This ties together systemd and etcd into a distributed init system. While fleet is still around, CoreOS has joined etcd with Kubernetes container orchestration to form an even more powerful management tool.

CoreOS also enables you to declaratively customize other operating system specifications, such as network configuration, user accounts, and systemd units, with cloud-config.

Put it all together and you have a Linux that’s constantly self-updating to the latest patches while giving you full control over its configuration from individual systems to thousand of container instances. Or, as CoreOS puts it, “You’ll never have to run Chef on every machine in order to change a single config value ever again.”

Let’s say you want to expand your DevOps control even further. CoreOS helps you there, too, by making it easy to deploy Kubernetes.

So, what does all this mean? CoreOS is built from the ground-up to make it easy to deploy, manage and run containers. Yes, other Linux distributions, such as the Red Hat family with Project Atomic, also enable you to do this, but for these distributions, it’s an add-on. CoreOS was designed from day one for containers.

If you foresee using containers in your business?—?and you’d better because Docker and containers are fast becoming The Way to develop and run business applications?—?then you must consider CoreOS Container Linux, no matter whether you’re running on bare-metal, virtual machines, or the cloud.

A version of this story was first published by Linode.

April 14, 2017
by sjvn01
0 comments

Big Linux bug, low security concerns

This Linux/Android bug sure sounded bad.

The National Institute of Standards and Technology (NIST) and Symantec announced a LinuxKernel ipv4/udp.c bug that made the LinuxKernel 4.4 and earlier vulnerable to remote code-execution. In turn, an attacker could exploit this issue to execute arbitrary code. Worse still, even failed exploits might cause denial-of-service attacks.

There’s only one problem with this analysis and the resulting uproar: It’s wrong.

Big Linux bug, low security concerns. More>