[Fsf-friends] Building a Community of GNU/ Linux Sysadmins for keeping watch over each other

[Ajay Pal Singh Atwal]

Hi

This is something i feel requires a little attention. So posting here.
Apologies for wasting your time, spelling mistakes, lack of knowledge, all
in advance.

I have been using FLOSS (GNU/ Linux) and related software on our servers
for around 5 years now. Most of the servers face Internet.

And as is the case with any evolving software even in GNU/ Linux(s) 
vulnerabilities are discovered and ofcourse patched form time to time.

But alas we ppl who manage servers are sometimes lazy, or forget and do
not patch servers in time. Alas we are only human.

And in general act as open invitations/ sitting ducks to crackers and
malicious ppl around the word.

I have seen this happen to a lot of ppl around, same has happened to me as
well, sometimes we are made to realize, by the crackers, that we havnt
been  vigil in our duties. And whatever experience I have in managing
server i still cant deny this fact that this can happen to me in future.

But hey we learn from mistakes and usually grow up. But what about
newbies, ppl whom we GNU/ Linux fans motivate, we ask them to switch over
to a better system from _you_know_what_$$_crap_they_are_using_

I still remember the old days when the number of script kiddies was much
less, atleast in India. Now ever Ram and Sham has Internet  access and is
eager to lay its hands on some script, trying to be the super kiddie.

And the poor newbie sysadmin we motivated to switch over to something
better, is an easy target. They becoem the victims, kiddies exploit their
servers. The machines are listed in XBL, RBL and in general cause
disservice to their users and others also. And the newbie sysadmin also
get the impression that GNU/ Linux is difficult/ insecure/ whatever (which
is not true). Usually we blame the newbie sysadmin that he havnt been too
vigil.

Most of such ppl remain clue less on what happened and why their systems
have misbehaved, what is happed to their servers. Some of such guys also
consider moving back to  _you_know_what_$$_crap_they_were_using_. And ppl
like us who motivated them are back to zero, all out time spent motivating
them goes down the drain.

Some of the glaring mistakes newbies make are Installing whatever version
of older unpacthed versions of GNU/ Linux they can lay their hands on.
AAnd then not configuring firewalls, not closing unnecessary services and
what not. All of this has been documented we all know that, but still
happens.

And all of the above is true for experienced sysadmins also. I have seen
so many websites being defaced/ mail servers being used for spam. After
all we are only human.

Cant we as a community of good ppl/sysadmins do something about it. Is it
possible to build a community where we can watch over each others back,
and report any problems in time to vulnerable system/ or systemes that are
already down the drain. And from community I do not mean another mailing
list or user group. Is it possible to do something automated, to keep
watch over servers, a distributed system. Where ppl who have subscribed 
to the system would have their system checked/ scanned periodically by
other systems, and sysadmin can be forwarned of existing/ new problems.
Something like an XBL, RBL but without the black list thing, but with a
warning to sysadmins.

Similar services are offered by some commercial vendors, but i believe a
community effort would be a better option, due to its very distributed
nature and scale. (More technicalities can be discussed later)

I am trying to forge such an alliance with two other sysadmins i know, and
hope something will come out of it. And we plan to make newbies, around
our area, part of it, and maybe help them with their newly setup servers,
so that they dont go back to _you_know_what_$$_crap_they_were_using_. Most
of it would be initially manual, except periodic port scans to locate
vulnerabilities, but later on more things can be automated.

If there is anything similar in place, or any advice or comments, please.

Please dont tell me that:
  * a good sysadmin dont need such a crap.
  * real sysadmins secure their systems like forts
  * real sysadmins dont make mistakes
  * pull out your network wire to secure your servers
  * RTFM
  * go away, you cant run a GNU/ Linux workstation, dont even think of
servers
  * blah blah


I think i have some experience, but still sometimes i need help and
confirmation that my servers are ok, what is wrong in third party
confirmations, if it is only a remove vulnerability scan.
And why not i can do that same for others and others can do it for me. I
have been doing this for 2-3 ppl already. And why cant we automate this
process and in a distributed manner.

What is my motivation for writing all this
==========================================

Recently i had some discussion with someone who is an advocate of FLOSS
and a dedicated GNU/ Linux user
pasting it here without his permission (this is part of an email discussion)
some parts edited/ changed

---------  snip  -------------
What ever you observe, that was correct. However, I am only user of
******* services. I forwarded your Email to concerned man on Friday

.. and he told me today (Monday), that system was hanged on Saturday
on rebooting, it fail to boot. In nutshell, there is *real problem* with
server. He is trying to fix it.
---------  snip  -------------

another part of email, some part edited

---------  snip  -------------
Before we start discussing, I would like to know frank opinion, about
FLOSS, is it going to help us.
---------  snip  -------------

Another discussion with someone else
parts of a telephonic conversation (whatever i can remember)
---------  snip  -------------
Him: The nameserver lookup is not working
Me: Have you checked the logs
Him: I cant, somehow the logs aint showing anything at all
Me: (Puzzled) that should mean, maybe server has been compromised
Me: (after a port scan of his machine, next day) there is sshd service
running on port 1422, you server is definitly compromised. Time to
reinstall.
---------  snip  -------------


And on Internet we can find a lot of machines which are either compromised
or ready to be compromised and we do nothing about them, ofcourse unless
the machines are honeypots and have been left like that intentionally.

Cant we help each other!!

Sincerely

Ajay Pal Singh Atwal
(Just Another GNU Users)