Monitoring User/Application Activity with psacct

One of the big advantages of using psacct on your server is that it provides excellent logging for activities of applications and users. When you are running scripts one of the important aspects of that script is how much resources it may be using and are there any resource limitations that may exist with the application. In addition, there may be times when you run a script as a user. In other words, you create a user with specific rights, maybe even using visudo. You will likely use this to reduce the security risks of a user who must issue a command with root privileges.

Monitoring User/Application Activity with psacct.

The Many Flavors of Linux

Even the smallest amount if research into Linux will have illustrated the sheer range of distributions out there. They are all based on the original Linux kernel built by Linus Torvalds (the father of modern Linux) and can all inter-operate to varying degrees. We have large selection of Linux distros on CD and DVD.

There are three base values that ensure all of the different distros will work together, and what application works for one should work for all.
Firstly there something called the Linux Standard Base. This is a set of specifications that the major distros adhere to which ensures interoperability between systems. It defines how packages are built and formatted, how file systems should work, and minimum criteria like applications and utilities to be available with the distro. This ensures that all Linux distros work from a common ground which makes sure everything will play nicely together.
There is also the Open Source Project. You will see this name a lot when you enter the world of Linux. Most distros include the same or very similar projects. Things like Apache, Samba, Mail, GNOME and KDE are all examples of Open Source Projects. The packages mentioned will work in whatever distro you have that conforms to this project.

Finally there is the Shell. This is the command line window most experienced Linux users seem to live in. It is the core of the machine/user interface and has to be standard across all distros. A command that works on one must be able to work on another.

I will list a few of the major distros currently in the world to whet your appetite. There are lots and lots out there and it would be impossible to do justice to all of them. If you are interested in them I heartily recommend doing some further research.

Debian
Debian is regarded as the largest Linux distribution to date in terms of users. It is renowned for its stability and quality and its interoperability with other software. It has a very large following and over 28,000 packages you can add to it. It is very stable and newbie friendly. The community is knowledgeable and helpful and most of the questions you may have will have been asked before on their forums. Debian seems somewhat over engineered to me, which isn’t a bad thing. It just means that it is never the latest and greatest, but is very, very stable. Currently on Debian Version 5, Lenny.

Ubuntu
Seemingly the most popular distro out in the world right now, since its release five years ago. Some suggest it has as much as 40% of Linux desktops. It is another good quality, stable release that is very newbie friendly. It even has a migration assistant for Windows refugees. This eases the pain a little by helping you in your first steps. Another distro that works with most hardware straight out of the box. Current version 9.04 Jaunty Jackalope.

Fedora
This is a distribution with a long history. Once part of Red Hat, Fedora was launched for the ‘Linux Hobbyist’. It is very secure and has a large number of packages available for it. This one seemed to be less newbie friendly than others, with not so many options for desktop users. If you want to run a web server or something then this would be ideal. Current version is Fedora 10, new version shortly

OpenSUSE
This is another distro with a rich history, now owned by Novell. It is a comprehensive package with some good documentation. The following is helpful and informative and there are lots of packages available for it. There is however some controversy over a deal it has with Microsoft, something about use of patents. This doesn’t detract from the usability of the product though. It does have graphical tools for an administrator. Current release is 11.1

Mandriva
Mandriva used to be Mandrake, and even as a Windows user back then I knew what that was. It is one of the distributions that offer both free and commercial versions of the software. The commercial version is regarded as one of the best distros for newbies. Although I don’t see that it has much that Ubuntu or Debian doesn’t offer and for free. Current version is Mandriva 2009.1.

Slackware
This is allegedly the oldest Linux distribution still around today. It is supposed to be the cleanest and most bug free edition available, but seems quite hard core. You definitely need skills to get into Slackware. It lags a little behind other distributions and doesn’t have all the cool bits to make your desktop smart. Current version 12.2

FreeBSD
Yes I know…it is not Linux but it is worth mentioning. It is primarily directed a server applications and is rated as one of the most stable. It doesn’t have all the bells and whistles that some of the more popular distros have but if you need something to run a server, you can’t go wrong with this. Current version 7.2.

Mint
This is my personal favorite, and not only for its cool name. It is based on Ubuntu, but is developed mainly ‘on demand’. The programmers have forums and feedback loops which they subscribe to. They listen and try to implement the best suggestions into the distribution. It has many specific tools, and to my mind is the most useable out of the box. Current version is Mint 6 Felicia.

PCLinuxOS
Despite the not so catchy name this distribution is supposed to be very good for the Linux newcomer. Another distro that works out of the box, natively supporting lots of hardware. The documentation and website are pretty good if a little disorganized. This one also seems to be English only, so if you speak something other, then you had best look elsewhere. Current version is PCLinuxOS 2009.1

Like I said, there are many other versions of Linux out there, and this is just a snapshot of the most popular. Now it’s up to you to choose your flavor…

What is Linux anyway?

Everybody knows Windows, and not always for the right reasons. Linux on the other hand can seem like dark magic only known to geeks and computer nerds to those of us on the outside. While it is harder to master than Windows, it is much more rewarding in that you finally gain control of your own PC. No more nagging from the operating system telling you it didn’t like what you just did, or not allowing you to delete that file you wanted to get rid of. If Linux tells you not to do something you can be sure it’s for a good reason!

Linux is an operating system that has its roots in Unix. It can do (almost) everything Windows can, and most of the time it can do it better. The fundamental difference between Windows and Linux is the approach. Windows is a commercial product designed to appeal to the widest possible audience and their various computers. It makes things as easy as possible to use, and hides all the workings from the user. This entails compromise.
Any system that tries to covers as wide a field as Windows isn’t going to be able to do everything well. To make it appeal and work for the widest audience the users have to be ‘protected’ from damaging the system and cede most of the control to it. Linux on the other hand has a more sensible approach. It has a core system which you can then bolt extras onto if you need them. You can tailor it to an amazing degree to your specific needs. It (mostly) isn’t a commercial product, and is maintained and developed by enthusiasts throughout the world.

The differenced I think can be summed up in one sentence. Windows was designed to sell, Linux was created to work.
Most distributions or ‘distros’ are free. You can download them, use them, abuse them, and pretty much do what you like with them. Most of the utilities and applications are also free. This is the other main attraction to new Linux users. Freedom. Not only do you not have to pay for any of the software, although I would encourage donations where appropriate, you have the freedom to do what you like. There are no lengthy Terms and Conditions to read, or copy protection because the software is free to use and distribute.

Linux main downside is that it is harder to manage to begin with. It can seem daunting at first, but well worth it once you get the hang of things. A lot of work has been done, and is being done to make the system easier for the newbie. The documentation is pretty good, and because the following of Linux is pretty fanatical there are hundreds of online forums and resources to help you along the way.
Linux, like your PC itself is made up of several components which I shall briefly describe below.

Processor
Even though you can see it, your operating system is doing several things at once. Most of them will have to go through your processor. Linux has a scheduler which prioritizes all the different demands and gives them to the processor in order. It decides what is important and what isn’t and ensures the processor deals with the important ones first.
Memory
Linux tries to use your available memory as much as possible as it works much faster than your hard drive. The processes mentioned above are stored here while they are being worked on. If you memory gets full then it has an overflow called Swap Space which pretends to be memory so your processor can use it.

Devices
Your PC is made up of any number of combinations of hardware. Sound cards, video cards, network cards, hard drives, the list is endless. Somehow Linux manages to support the majority of them while remaining a decent size. It does this by using modules. These are like ‘bolt-ons’ that can be added if they are needed by the system. This allows the flexibility to add and remove hardware as you need to.
Kernel
The kernel is the heart of Linux. It is the sun around which everything orbits and receives life. This is what makes Linux what it is. It is a core program that controls everything around it. It coordinates everything that goes on within the system.
File System
Where you had FAT and NTFS in Windows, here we have ext2, ext3 and others. Think of them as a library. Your hard drive is the book shelf, and the file system is how the books are arranged. There has to be a system for you to quickly find your book and the same for computer files.
GUI
The Graphical User Interface (GUI) is the pretty desktop you see. Windows is purely GUI driven, in that everything you do is dragging and dropping from one window to another, which is where it got its name. Although Linux can do the same, it works differently. At its core Linux is controlled from the command line. You type in commands and the machine does its thing.
Applications
These are software addons you can install to achieve a certain goal. Things like music players, office tools, web browsers etc. Exactly the same as buying a firewall or office suite for Windows, except for the most part, applications for Linux are free. There are commercial applications out there which you have to pay for, but they are in the minority, and unless your needs are extremely specific, not really necessary.
Utilities
These are similar to applications in that you can install them on your system for a specific purpose. However that purpose is very different. Applications are like productivity tools, they help you produce something. Utilities help you manage your system like managing hard drives, monitoring networks, firewalls and that kind of thing.
That is your Linux system in a nutshell. As with everything, there is as much technical detail as you could wish for if that’s your thing, but this was just an overview to familiarize you with Linux. Pretty straightforward once explained isn’t it?

America’s Schools: Held Hostage by Microsoft

Schools in America are held hostage by Microsoft and are being choked one school at a time. We have allowed Microsoft and our State and Federal governments to force us into financing cycles that schools cannot afford.

I have lived in Montana most of my life and worked with schools to get technology funding for 14 years now. The districts I work with use Linux on 75% of the Desktops and all of the servers. I have pushed Open Source applications and operating systems in schools for a long time and there is some progress. However, Microsoft has a serious choke hold on schools. Here is how the choke works.

1. Unlicensed Software Installs
One way schools get in trouble is when they feel they do not have enough money so instead of purchasing licenses they install more products without the proper license. To Microsoft, this is piracy and theft. I agree with Microsoft here, if they have a product and a price you either pay or do something else. I am amazed when I hear schools that steal software from Microsoft but then tell kids there are rules and ethics that they must follow. Schools that cheat on licenses are placing themselves at the mercy of Microsoft.

2. Software Updates
Software purchases must always be made with the recognition and the funding that the software must be updated both in terms of license and hardware. Here is how this works. A school gets a grant for technology and purchases Library software in year one. Not only must the school continue to purchase Library software updates each year, they must also purchase the Microsoft license that allows you to run the software. Oh…and also you must purchase the hardware that runs the software for Microsoft. This is called the Domino effect…one purchase forces the next purchase and schools are locked into these scenarios where they cannot get out. Microsoft controls the software, the cycle and the cost of schools teaching students.

3. State and Federal Requirements
One of the most damaging trends for schools in America is the demand that they purchase Microsoft products in order to report to Federal and State authorities. The State of Montana requires that you provide information from the school to an online sources for the state.  The Office of Public Instruction says that they are creating an online interface for schools that is easy to use and will facilitate all operating systems.  Well,only if your operating system is Windows (at least Windows 2000) and Internet Explorer 6-7. If you use Linux you cannot report to the state.

Schools and Libraries is a site used by the Federal government to provides funding so schools can get Internet access and help with technology. However, you cannot apply for this Federal funding, provided especially for poor schools, unless you have Internet Explorer 6-7.

What is worse is that each year the requirements for school reporting are tied to State and Federal funding outside of technology and the technical requirements to make those reports is tied to the updated versions of Windows and Word or Excel. I talk with people involved in schools in Europe on a weekly basis and they always talk about the freedom that the government allows in terms of which operating system and applications are used in schools.

What’s up with the U.S.? Why are schools locked into these strangle holds?

Long live Open Source!

Finding Changes with Timestamps

Timestamps are important in terms of diagnostics because they provide you with information about when an event occurred. For example, when a file was last changed. This information alone is invaluable for troubleshooting. The timestamp represents the last time the file was modified. As you look at these examples, you can see they all have different modification dates.
Dates. The file training was modified 2000-07-05 at 19.01 and the file squid.rtf was modified 2008-05-27 at 16:24 so you can see both date and time are available.
-rw-r–r– 1 mike mike 30722 2008-07-05 19:01 training
-rw-r–r– 1 mike mike 997 2008-05-27 16:24 squid.rtf

In these examples you see a directory, notice the “d” at the start of the line. The dates of this same directory are different because a sub-directory was changed in test so the second example reflects that change.
drwxr-xr-x 3 mike mike 4096 2008-08-13 16:47 test
drwxr-xr-x 4 mike mike 4096 2008-08-14 09:21 test

A powerful utility for locating changes is the find command. With the Linux find utility, you can perform powerful searches on just about any criterion you can think of, and then–from the same command-line entry–invoke another utility to do whatever you need to do with the results.
In order to perform the most basic of searches, you’ll need to specify two things:
The search path–You can perform a search in either a specific path, or the entire filesystem. Since find is inherently recursive, the search will automatically extend to all of the subdirectories beneath of the directory that you specify.
What you’re searching for–There are a lot of ways that you can specify this. You can search for files of a specific name, and decide whether to make the search case-sensitive. You can also use wildcards, or search for files with certain characteristics or that are of a certain age. Or, you can combine multiple criteria for even more specific searches. The main thing that limits you is your own imagination.

So now, for example if you want to search the entire filesystem for all files whose names end in “.conf”. You’ll want to use either the “-name” or the “-iname” switch in front of the file description that you want to search for. Otherwise, you’ll get a jumbled up mess of every directory listing that you’ve searched, with the information you’re looking for mixed in. For case-sensitive searches, use “-name”; for case-insensitive searches, use “-iname”. In this case, use “-iname”, since you want to make the search case-insensitive. If you include a wildcard character in with a search criterion, you’ll need to enclose that search criterion in quotes. That will keep the shell from interpreting the wildcard character as an ambiguous file reference.

sudo find / -iname ‘*.conf’
—cut—
/etc/vsftpd/vsftpd.conf
/root/vsftpd.conf
—cut—
In reference to timestamps what you are really interested in are two requirements, the name of the file and the time it was changed.

You can perform searches with more than one search criterion. If you separate the criteria with a space, it will be the same as placing an “and” operator between them. The “-mtime -7″ switch to find all of the “.conf” files that were modified within the last seven days.

sudo find / -iname ‘*.conf’ -mtime -7
/etc/mplayerplug-in.conf
/etc/awstats/awstats.model.conf
/etc/awstats/awstats.localhost.localdomain.conf
/etc/httpd/conf.d/awstats.conf
/etc/yumex.profiles.conf
/var/cache/yum/yumex-mirror-cache.conf
—cut—

There are several important parts to find. First you will need to use sudo to be able to access many files as they will be owned by root. Use the find command followed by the directory you want to search. If you want to search the whole server use “/”. If you want to limit your search to a specific directory indicate that. Using the -iname is the easiest option as it allows all files regardless of case.

The important part is looking for a text string which must be enclosed in single quotes. A common wildcard is to use the “*” indicating it will match anything. So ‘*.conf’ will match any file that ends with “.conf”. If you use ‘*.*’ it will be a wildcard for anything. The -mtime is a search for files modified within a time period.

Here are some practical examples:

Files on the system modified within the last 24 hours. Note this is looking for all types of files on the whole system.

sudo find / -iname ‘*.*’ -mtime -1

Files that have changed in the /var/www directory in the last week.

sudo find /var/www -iname ‘*.*’ -mtime -7

Files that have changed in the apache web server configuration directory in the last 14 days.

sudo find /etc/apache2 -iname ‘*.*’ -mtime -14

You can change your search with find to locate files that have been accessed within a time period. For example if you wanted to locate files that have been accessed in the /usr/share directory you could use this command:

find /usr/share -iname ‘*.*’ -atime -1

That would list files accessed by a user or the system within the last 24 hours in the /usr/share directory.