The Mailbag

HELP WANTED : Article Ideas
Submit comments about articles, or articles themselves (after reading our guidelines) to The Editors of Linux Gazette, and technical answers and tips about Linux to The Answer Gang.

reminder software
Security implications of root login over SSH

reminder software

Wed, 14 Sep 2005 18:00:35 +0530
J.Bakshi (hizibizi from spymac.com)
Answered By Thomas Adam

Hi all,

I like to add a reminder software with my system. I have tested calendar and am not satisfied with it. now I have very sophisticated reminder tool called remind . its tkremind front end adds special weight to remind by making configuration as easy as Kalarm of KDE. I am looking for a tool which can make the message (from remind) appear on my desktop. could any one give any idea ?

[Thomas] Assuming remind uses some transient form to store the reminder somewhere, then you can use 'osd_cat' or some other utility to have it "display" on the screen.

I have found another tool called xmessage -

[Thomas] That has been around since the 80s.

like

xmessage -file  <filename>

so you have to write your message in a file (ascii) first.

[Thomas] You don't even need to do that. xmessage can read from STDIN as well.

Security implications of root login over SSH

Mon, 12 Sep 2005 20:11:20 -0700 (PDT)
Riza Aziz (telefonixar from yahoo.com)
Answered By Kapil Hari Paranjape

Hi there,

I'm wondering if it's wise to allow a remote user within the LAN to log in as root, by adding that user's public key to root's "authorized_keys" for that machine.

[Kapil] There is an "sudo"-like mechanism within SSH for doing this. In the authorized_keys file you put a "command=...." entry which ensures that this key can only be used to run that specific command.

All the usual warnings a la "sudo" apply regarding what commands should be allowed. It is generally a good idea to also prevent the agent forwarding, X11 forwarding and pty allocation.

Here is an entry that I use for "rsync" access. (I have wrapped the line and AAAA.... is the ssh key which has been truncated).

from="172.16.1.28",command="rsync -aCz --server --sender . .",
no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty
ssh-dss AAAA..... rsyncuser

I'm writing some scripts to back up data on our small business network here. One option is to get each machine to periodically dump its data on a specific machine using NFS. The option I'm interested in is to get a designated machine to remotely login to each machine and transfer the files over a tar-ssh pipe.

The only reason to be using root access is because some directories (/root, some in /var/lib) can only be read by root. Would changing permissions (e.g. /var/lib/rpm) affect anything, if I chgrp the directories to a "backup" usergroup?

I'm concerned with one machine, a web server, that will be included in the backup scheme. All machines here use Class A private network addresses and are behind a NAT firewall, but the web server can be accessed from the Internet. Will allowing root login over ssh on that machine pose a huge security risk, even by allowing ssh traffic from only the local network?

GENERAL MAIL

/LDP/LGNET/114/misc/tag/mike.show-mtime.pl.txt
Outlook replacements --or-- Five reasons NOT to use Linux
Jabberd Installation Guide Comments
Localhost considered harmful
LG 105: RSS and Feed Readers Addendum

/LDP/LGNET/114/misc/tag/mike.show-mtime.pl.txt

Sat, 27 Aug 2005 15:18:58 -0400
Tony Dodd (tony from wefixtech.co.uk)

[Rick] I just happened to be reading the main discussion list at the Linux Documentation Project, and saw this post that appears intended for the Gazette, instead.

It seems to be intended as a correction to http://linuxgazette.net/114/misc/tag/mike.show-mtime.pl.txt .

[Ben] Thanks, Rick - that's great. I'm going to CC Tony on this; perhaps he'll find it useful.

Hi All,

There's an issue with the above file - that just caused my syslog to jump from about 5mb to 150mb.

"Use of uninitialized value in numeric ne (!=) at mike.show-mtime.pl.txt line 7"

If someone could patch it, i'd be grateful

Diff:

--- mike.show-mtime.pl.txt.orig   2005-08-27 07:43:18.000000000 -0400
+++ mike.show-mtime.pl.txt        2005-08-27 07:43:25.000000000 -0400
@@ -3,7 +3,7 @@
 my( $a, $b ) = 0;
 {
 $b = ( stat "foo" )[ 9 ];
-if ( $a != $b ){
+if ( $a ne $b ){
 print scalar localtime, ": $b";
 $a = $b
 }

[Ben] Hi, Tony -

(Even though the script is credited to Mike Orr, I'm the author of it - it just got misattributed. I don't expect him to answer for my sins.

The problem with the above is that you don't have a file called "foo"; as a result, 'stat "foo"' comes out empty, and the rest of the problem follows from that. The above was never meant to be a complete script, simply an example of the algorithm for Suramya (I seem to recall that's who started the original discussion) to use.

I agree that changing "!=" to "ne" would "fix" the problem - but it would be the wrong problem. Pointing the script at an existing file - perhaps by changing 'stat "foo"' to 'stat $ARGV[0]' and specifying the filename on the commandline - would seem to me to be the "right" solution. Of course, the rest of the script should be rewritten to meet real-life conditions as well... but that's beyond the scope of what we were talking about.

Outlook replacements

Tue, 30 Aug 2005 19:57:30 +0100
Jimmy O'Regan (The LG Answer Gang)
Answered By Mike Orr, Heather Stern

http://www.linux-watch.com/news/NS8124627492.html

...............

Reason number one: Linux is too complicated

Even with the KDE and GNOME graphical windowing interfaces, it's possible -- not likely, but possible -- that you'll need to use a command line now and again, or edit a configuration file.

Compare that with Windows where, it's possible -- not likely, but possible -- that you'll need to use a command line now and again, or edit the Windows registry, where, as they like to tell you, one wrong move could destroy your system forever.

...............

[Sluggo] When people say "Linux doesn't have enough applications", that usually translates to, "Linux doesn't have certain specific applications, namely MS Office, Photoshop, Yahoo Messenger with webcam, etc."

[Heather] StarOffice, (ok photshop is a fair dig, we have lots of things like it, but none are trying hard to be it), yahoo with webcam == ayttm but most people don't know that.

In a particularly ahem lively discussion at the starport last week, it was agreed that the problem is that menu interface standardization isn't. Not even by Mr.Tanenbaum's broad definitions.

For example RH has been using Gnome for awhile, but has changed the menu layout in every major revision and some of the minor ones.

K and Gnome fans alike can't decide whether to keep their menus top or bottom. If they were hidden you'd have a flip a coin chance of even knowing where to look.

To most newbs "click the root menu" may as well be hidden entirely, because they won't really think that's useful if they have mswin experience (it pulls up display settings) and if they don't have even that they're just plain lost.

[Sluggo] Interestingly, I was going to mention Outlook and Outlook Express, but I haven't heard much about them recently. Has their popularity diminished?

[Heather] Yes, and thunderbird's and eudora's have increased.

My friend Colleen is looking to start an article series on people starting from zero* into Linux. I will of course be encouraging her and helping her out, too.

* yeah, absolute zero, Kelvin. The kind of people who think "my god, at least something says where to start" when they look at Windows(tm), then are stalled because they're afraid of the rest of the menu.

Much as I didn't find linspire groovy, not to my beat, daddy-O, it serves an important duty for some.

Outlook Express comes as default with Windows, and it's not a particularly good email client. Thunderbird is taking that market.

Outlook is another kettle of fish: it isn't about using Outlook, it's about accessing an Exchange server. The big news back when Novell bought Ximian was that they open sourced Ximian Connector, so Evolution could access Exchange servers. I had a look through the code, and... it's a hack, basically.

Outlook and Exchange communicate with an extremely complicated protocol. Ximian Connector just connects to the Exchange web interface, if it's available, and basically acts as a screen scraper (not exactly: it works using a modified version of WebDAV, but it also screen scrapes to get enough data to be useful).

People who run versions of Exchange that don't have a web interface still have to stick to Windows. People who don't have to pull to get the web interface set up are also out of luck.

On the server end, it's not too bad. There are several open source Exchange alternatives that have equivalent features. The Outlook Connector Project (http://openconnector.org) aims to provide an open source set of MAPI DLLs to be used by open source projects (such as Kolab (http://kolab.org) or Open-Xchange (http://www.openexchange.com)), so Outlook can connect to them. Once the server end has been migrated away from Exchange, it's possible to bring in Linux at the client end with little disruption.

There's also work being done towards implementing the actual protocols used between Exchange and Outlook. Luke Leighton, formerly a Samba developer, has reverse engineered most (if not all) of the protocol (http://www.winehq.org/hypermail/wine-devel/2005/01/1054.html), and has started work on both client and server software (http://cvs.sourceforge.net/viewcvs.py/oser/exchange5.5).

The OpenChange (http://openchange.org) project is also working (slowly) towards an Exchange replacement. They seem to be focusing more on reverse engineering the database format used by Exchange, so there isn't too much overlap (so far).

Jabberd Installation Guide Comments

Tue, 06 Sep 2005 13:41:27 -0700
Adam Pilorz (adam from pilorz.net)
Answered By Suramya Tomar

[Suramya] Tag,

Got the following feedback on my Jabber install guide. It has some good advice for improvements so...

I've just installed Jabberd 2 server following your Guide (http://linuxgazette.net/112/tomar.html), and I want to share some experiences which could improve this guide.

[Suramya] Thanks for taking the time to email me with your feedback. Would you mind if I share this with the Linux Gazette so that they can publish it in their next issue. That will help other people who are trying to install a Jabber server.

No, of course I don't mind . You can share it with everyone, I've wrote it only to help others .

1) Since Jabberd 2.0s3 or Jabberd 2.0s4 (the newest one is 2.0s9) the Libidn library isn't installed automatically with Jabberd, so you should write it in your guide, that it must be installed BEFORE running ./configure in jabberd directory. It'll save time of people trying to follow your Guide.

[Suramya] Ok. I didn't know that. Thats a good thing to keep in mind for future releases.

2) It would be easier if you wrote some information on installing MySQL libraries, because I had to use --disable-mysql option, as I couldn't find right libraries (according to some mailing list mysql-client and mysql-devel libraries) and/or write, that it isn't a must - user can replace mysql by barkeley in c2s.xml configuration file.

[Suramya] hmm. I didn't put instructions on how to install mysql because http://mysql.com had instructions on how to install the mysql libraries. But thats something to keep in mind for the next version I guess.

Yeah, you can replace MySQL by Berkley DB, but I havn't set it up using that so I don't know how...

About MySQL - well, I've asked for it, because I had a lot of trouble with MySQL. At last I've menaged to install Jabberd 2 with MySQL after writing last letter to you. I've configured it with options

./configure CPPFLAGS=-I/usr/include/mysql LDFLAGS=-L/usr/lib/mysql --enable-debug

It looks like Jabberd 2 can't find MySQL even if it's installed (I've installed RPM's from my Red Hat install CD). So I think it would be a good idea to write something about CPPFLAGS and LDFLAGS options for those, who would have same problems as I had. It would be also useful, if you put some info on how to create SSL certificate for server (I've followed those instructions: http://jabberd.jabberstudio.org/2/docs/app_sslkey.html to create SSL Certificate and those: http://jabberd.jabberstudio.org/2/docs/section05.html#5_2 to set up Jabberd2 to use them).

I think that's all I had to write for now. Thanks for your answer.

3) Also, it would be a useful suggestion to enable debugging (run ./configure --enable-debug), so everyone could see what's going on, when something goes wrong.

[Suramya] Good suggestion. I will definitely add that in the next version.

By the way I want to thank you for such a good installing guide

[Suramya] Thanks. Glad you liked it.

Localhost considered harmful

Sun, 11 Sep 2005 22:42:47 -0700
Rick Moen (LG Contributing Editor)

OK, guys, this is a bit wacky. Somebody's going to have to telephone T.R., to advise him of a troublesome error in the genetikayos.com DNS, which in turn is clobbering deliverability of e-mail to hostname "linuxgazette.net". As a reminder, our primary DNS is at T.R.'s machine, and my nameserver pulls down copies from there as our second of (only) two nameservers.

T.R. is at [snip]. I've been trying to call him, but his line's been busy.

First thing I noticed, a few minutes ago, was bunch of SMTP error messages like this:

----- Forwarded message from Mail Delivery System <Mailer-Daemon@linuxmafia.com> -----

From: Mail Delivery System <Mailer-Daemon@linuxmafia.com>
To: tag-bounces@lists.linuxgazette.net
Subject: Mail delivery failed: returning message to sender
Date: Sun, 11 Sep 2005 21:42:52 -0700

This message was created automatically by mail delivery software.

A message that you sent could not be delivered to one or more of its
recipients. This is a permanent error. The following address(es) failed:

  ben@[snip]
    all relevant MX records point to non-existent hosts

------ This is a copy of the message, including all the headers. ------

[Snip a copy of Jimmy's post to tag@, as addressed by my mailing list software to Ben's subscription address of [snip] .]

OK, so next step was to remind myself of what are our MX records, because I couldn't remember:

[rick@linuxmafia]
~ $ dig -t mx linuxgazette.net

; <<>> DiG 9.2.4 <<>> -t mx linuxgazette.net
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 38278
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 3

;; QUESTION SECTION:
;linuxgazette.net.              IN      MX

;; ANSWER SECTION:
linuxgazette.net.       41736   IN      MX      10 genetikayos.com.

;; AUTHORITY SECTION:
linuxgazette.net.       41720   IN      NS      ns1.linuxmafia.com.
linuxgazette.net.       41720   IN      NS      ns1.genetikayos.com.

;; ADDITIONAL SECTION:
genetikayos.com.        1218    IN      A       127.0.0.1
ns1.linuxmafia.com.     139310  IN      A       198.144.195.186
ns1.genetikayos.com.    128120  IN      A       64.246.26.120

;; Query time: 53 msec
;; SERVER: 198.144.192.2#53(198.144.192.2)
;; WHEN: Sun Sep 11 22:02:21 2005
;; MSG SIZE  rcvd: 160

Some of you will be really sharp-eyed and immediately spot something eye-popping in the above (especially since I've called your attention to the possibility) -- but put yourself in my shoes, and imagine that all you saw, at first, is the single MX record, priority=10, pointing to genetikayos.com .

Ah, that makes sense: Mail to our domain other than mail to our mailing list subhost gets redirected to T.R.'s machine. So, I absent-mindedly carry out the standard next step in diagnosing SMTP problems, which is to attempt a manual SMTP session using /usr/bin/telnet -- and I got a heck of a big surprise:

[rick@linuxmafia]
~ $ telnet genetikayos.com smtp
Trying 127.0.0.1...
Connected to localhost.localdomain.
Escape character is '^]'.
220-linuxmafia.com ESMTP Exim 4.44 (EximConfig 2.0) Sun, 11 Sep 2005 22:02:53 -0700
220-.
220-WARNING:  Unsolicited commercial e-mail (UCE/SPAM), pornographic
220-material, viruses, and relaying are prohibited by this server and
220-any such messages will be rejected/filtered automatically,
220-depending on content.
220-.
220-By using this server, you agree not to send any messages of the
220-above nature.  Please disconnect immediately, if you do not agree
220-to these terms and conditions.
220-.
220-Please contact postmaster@linuxmafia.com if you have any
220-enquiries about or problems with this server.
220-.
220-Find out more about EximConfig for the Exim mailer by visiting
220-the following URL:  http://www.jcdigita.com/eximconfig
220 .

Er, what? That's my SMTP banner. Wait, didn't I just telnet into _T.R.'s_ SMTP port?

At this point, I gazed a bit higher up, re-read the "dig" output, and boggled:

; <<>> DiG 9.2.4 <<>> -t mx linuxgazette.net
[snip]

;; ANSWER SECTION:
linuxgazette.net.       41736   IN      MX      10 genetikayos.com.
[snip]

;; ADDITIONAL SECTION:
genetikayos.com.        1218    IN      A       127.0.0.1
                                                ^^^^^^^^^

Um, OK. That now gets appended to the Big Book of Things Not to Do with DNS.

Just to double-check:

[rick@linuxmafia]
~ $ host genetikayos.com
genetikayos.com has address 127.0.0.1

There are times when the loopback address is just not your friend. Writing SMTP-related DNS RRs is one of those times.

I'll probably keep trying to reach T.R. by telephone for a while: He's likely not very reachable by e-mail at the moment.

[Jimmy] Eight hours later...

T.R. has fixed his DNS

[rick@linuxmafia]
~ $ host genetikayos.com
genetikayos.com has address 64.246.26.120

LG 105: RSS and Feed Readers Addendum

Thu, 22 Sep 2005 11:16:02 +0100
Jimmy O'Regan (The LG Answer Gang)

(This came up elsewhere. I remember grinding my teeth about it at the time, so I don't know why I didn't put it in the article.)

As well as having 10 different versions[1], RSS has two competing time formats: RSS 0.91/2.0 use RFC 822 (date -R) format, RSS 1.0 uses dc:date, which needs W3CDTF (a subset of ISO 8601: date --iso-8601, date --iso-8601='minutes', date --iso-8601='seconds')

Mark Pilgrim has a blog entry that explains a bit of the history behind this here: http://diveintomark.org/archives/2003/06/21/history_of_rss_date_formats

[1] I said 9 in the article, but RSS 3.0 came afterwards (and, IIRC, is nothing like the other 9 RSS formats)

GAZETTE MATTERS

Heather is in England...
Heather and Jim taking flight.
SVN

Heather is in England...

Thu, 22 Sep 2005 21:16:10 +0100
Jimmy O'Regan (The LG Answer Gang)

...and so I have the keys to the mailbag this month (errors, etc., are my fault).

Heather has gotten hold of an English mobile phone, on loan from a friend, and is trying to get to grips with both SMS and being a tourist...

[Heather] OH BTW, ENGLAND IS NICE!

[Jimmy] Try to spend as much time as possible being a tourist!

[Heather] A POOR TOURIST, BUT STILL!

[Jimmy] Oh heck, all you have to do is photograph everything you see, and ask the most obvious questions that come to mind. It's fun, give it a whirl.

[Jimmy] Oh. Can't send you photos.

[Heather] CAPS ARE THE CLUE, THIS PHONE IS PRETTY OLD...

[Jimmy] Ah. Large and brick-shaped, prompting the question: do I drive this?

[Heather] IF YOU HAVE 2 ASK YOU CAN'T AFFORD IT :-P

Heather will be back in time for next month's issue.

Heather and Jim taking flight.

Sat, 17 Sep 2005 06:54:52 -0700
Heather Stern (The Answer Gang's Editor Gal)

We'll be heading off to the UK this weekend. Jim reports that we've been signed up for internet access at his hotel assignment, so I'll probably be able to send in a blurb, but hopefully I will have enough things to do that I won't be lurking in a hotel all that much.

[Ben] [smile] Enjoy yourselves and don't let your vacation time be spoiled by schedules. If you can get it in without stressing, cool; if not, there's always next month.

[Thomas] Indeed, although I am sure something can be arranged.

My poor mactop decided to have a fit when I tried to make it dual boot. One of the LUG locals is a truly guru macintosh fella though, so he's done his best to fix it up, and I'll have it back at the installfest. If it's not up to speed I'll end up taking terra with me instead... er, as soon as I seal her up. those lil bitty screws are kinda necessary after all if you don't want people gettin' weird because your laptop's falling apart. To its credit terra's hibernation works perfectly

Anyways there won't be cover art unless I'm taking the mac.

I've crosstrained Ben in using the g2 form of lgazmail to generate tips and the mailbag parts, by way of showing him the generation-phase on issue 118's data.

[Ben] FSVO "trained". You certainly gave it your best shot; the rest is up to that gadget I carry around between my ears. Don't worry, if it all falls down around my ears and everyone hates me and the government sends in the black helicopters, I promise to not blame you with my last dying breath.

Of course everyone won't hate you. We've got xteddy in the tag lounge.

In short, if ben would like to login to gemini he should be able to play virtual Heather this month. Hopefully that should serve in case I happen to be truly without 'net.

[Ben] Yeesh, options. I feel my brain melting already...

Bye now! See you next month!

[Ben] Enjoy the trip, both of you.

Oh yeah, someone tickle silentk; I think he got some more kudo letters

[Ben] Presumably, there's a "someone" here who knows how to do that. I hope.

[Thomas] You need a cattle-prod.

[Ben] [blink] I usually find that a nice cup of coffee does it for me, but I'll take your suggestion under advisement. (Now that I think about it, I've had a few mornings when it would probably have been just the thing...)

SVN

Thu, 08 Sep 2005 19:19:03 -0400
Benjamin A. Okopnik (ben from linuxgazette.net)

Kayos has just updated the SVN back-end to FSFS. Hopefully, this will result in fewer deadlocks - I guess we'll find out as time goes on.

[Jimmy] Sure enough, there have been fewer problems (so far) this month. Hopefully the trend will continue.

This page edited and maintained by the Editors of Linux Gazette
HTML script maintained by Heather Stern of Starshine Technical Services, http://www.starshine.org/

Published in Issue 119 of Linux Gazette, October 2005

More 2 Cent Tips!

See also: The Answer Gang's Knowledge Base and the LG Search Engine

closing unneeded ports
TFTP problem
HTTPS question
Mozilla hogging the screen
Couple of perl related questions
Securing rsync
Boot from cd via floppy
Security implications of root login over SSH

closing unneeded ports

Mike Orr (LG Contributing Editor)
Answered By Kapil Hari Paranjape, Peter Knaggs, Predrag Ivanovic, Thomas Adam

While most people know to turn off any services they don't want to offer the world, many do not realize this applies at the interface level as well as the service level.

[Kapil] Other than configuring this by editing the configuration files for the individual daemons that open the listening sockets, you can also use iptables/ipchains to block the (ir)relevant address/port pairs.

Here is the relevant portion of a file called "iptables.save" on a machine that runs a public web server and also accepts ssh connections.

*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -d 127.0.0.1 -i lo -j ACCEPT
-A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 80 -j ACCEPT
COMMIT

You can enable this with

	iptables-restore < iptables.save

You can add/remove ports according to what connections you wish to accept. You should probably also accept some icmp connections in order to avoid losing routing information.

A typical networked computer has two interfaces: lo (the loopback) and eth0 (the Ethernet). Most daemons listen on all interfaces unless you tell them otherwise. Obviously, your web server, mail server, and CUPS (printer) server must listen on the public interface if you want other computers to access them. But if you're running mail or CUPS only for your own computer, you should make them listen only on the localhost. This eliminates a bunch of security vunerabilities because inaccessible programs can't be exploited.

There are several portscanners available but good ol' netstat works fine if you're logged into the target computer.

# netstat -a --inet
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 *:631                   *:*                     LISTEN
tcp        0      0 *:https                 *:*                     LISTEN
udp        0      0 *:bootpc                *:*
udp        0      0 *:631                   *:*

Add the "-n" option to bypass domain-name resolution. Here we see the secure web server listening on all interfaces (*:https), good. But CUPS is also listening on all interfaces (*:631, both TCP and UDP), bad. (We know port 631 is CUPS because that's what we type in our web browser to access the admin interface.) To make CUPS listen only on the localhost, I edited /etc/cups/cupsd.conf, commented the "Port 631" line and added "Listen localhost:631". (I like Listen better than Port because it shows in one line exactly which host:port combinations are in effect.) Note that you can't specify an interface directly, you have to specify the domain/IP attached to that interface.

Then I restarted the server and checked netstat again:

# /etc/init.d/cupsd restart
 * Stopping cupsd...
[ ok ]
 * Starting cupsd...
[ ok ]
# netstat -a --inet
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 localhost:631           *:*                     LISTEN
tcp        0      0 *:https                 *:*                     LISTEN
tcp        0      0 10.0.0.1:32775          example.com:imaps ESTABLISHED
udp        0      0 *:bootpc                *:*
udp        0      0 *:631                   *:*

Good, the TCP line changed to "localhost:631". The UDP line is still "*:631". I searched the config file and "man cupsd" for "udp" but found nothing. I guess that means you can't turn it off? I decided not to worry about it.

There's a new line in netstat: "10.0.0.1:32775 to example.com:imaps". It looks like Mozilla Thunderbird is automatically checking for mail. 10.0.0.1 happens to be my public IP. (IPs/domains changed to protect the innocent.) It connected to the secure IMAP port on example.com. 32775 was a free port the kernel chose at random, as always happens when you connect to an external server.

There's still one suspicious line, "*:bootpc". I'm not running a diskless workstation or doing any exotic remote booting, so what is this? "lsof" is a very nifty program that tells you which process has a file or socket open.

# lsof -i :bootpc
COMMAND  PID USER   FD   TYPE DEVICE SIZE NODE NAME
dhcpcd  3846 root    4u  IPv4   5398       UDP *:bootpc

I am using DHCP, which runs this daemon while you're leasing an IP. I ran "man dhcpcd" and searched for "bootpc" and "port". Nothing. I guess it uses that port for some unknown reason. I decided not to worry about it.

[Kapil] Not quite. You shouldn't be running the dhcp-server (which is what the dhcpd program is). You are using dhcp in client mode so you should disable dhcpd from starting up.

[Peter] True, but the program in question listening on the UDP port 68 (bootpc) is "dhcpcd", not the dhcp-server which indeed has the name "dhcpd". When a client requests a DHCP address, a proccess (either "dhclient" or "dhcpcd") listens on UDP port 68.

It's eleven o'clock. Do you know which services your computer is running?

[Pedja] OK, what about this?

pedja@deus:~ ]$ netstat -a --inet
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0                *:6000                       *:*                     LISTEN

That's X server, right?

root@deus:/home/pedja#lsof -i :6000
COMMAND  PID USER   FD   TYPE DEVICE SIZE NODE NAME
X       3840 root    1u  IPv6   9813       TCP *:6000 (LISTEN)
X       3840 root    3u  IPv4   9814       TCP *:6000 (LISTEN)

I should add something like 'tcp -nolisten' to options that are passed to X when it starts(I use startx to,well,start X ). My question is where to?

[Thomas]

/etc/X11/xinit/xserverrc

Is the file you're looking for. By default (on most distros, anyway), the '-tcp nolisten' are set already.

[Pedja] There's no xserverrc in Crux, so I made one with

#!/bin/sh
exec /usr/X11R6/bin/X -dpi 100 -nolisten tcp

in it. I've put it in my home folder.

[Pedja] Should I make an alias in .bashrc,like

startx  () { /usr/X11R6/bin/startx -- -dpi 100 ${1+"$@"} 2>&1 | tee $HOME/.X.err ; }

or modify .xinitrc in ~, or... What's The Right Thing(tm) to do?

[Thomas] No alias. See above.

TFTP problem

mishradk (mishradk from vsnl.net)
Answered By Breen Mullins, Ben Okopnik

Hi,

I am a starter in GNU/Linux. I am using Linux Kernel 2.4.20-8 Redhat Linux 9.

I have written a TFTP client and server. I have created a UDP socket and as per the RFC i am sending a structure with the proper TFTP header and then data.

it is working fine and i am able to send and get files.

my problem is when i use ethereal and tell to capture the TFTP and specified port it shows that the packets are UDP + data. I think i should get UDP header , then TFTP header and then data. But this is not happening in my case. My TFTP header is also coming as data.

How can I solve this problem...

[Breen] You're not by chance using a non-standard port for your tftp server, are you? If the traffic isn't on port 69/udp, ethereal won't know to decode it as TFTP.

[Ben] I think that your best bet would be to look at a standard TFTP conversation and compare it to yours. There may be some subtle difference that you're missing, or perhaps a part of the RFC that you're misinterpreting.

I dont have any guide.. hope to get a reply and from you people.

[Ben] I have not read it myself, but I understand that Richard Stevens' "UNIX Network Programming" series is the classic reference for this kind of work.

Hi Breen

you are right.. i had used a non std port. so it was not showing it as TFTP.

[Breen] Hi Deepak --

I've got two requests:

1) Please don't post html. Email is a text medium.

2) When you ask a question on a mailing list, you should follow up on the mailing list. That allows all subscribers to benefit from the answer you receive. I've added The Answer Gang back to the recipients of this email.

Glad we were able to help you!

HTTPS question

Mike Orr (LG Contributing Editor)
Answered By Jay R. Ashworth, Brian Bilbrey, Ramon van Alteren

Is there any way to have multiple HTTPS domains on the same IP/port? The mod_ssl FAQ says name-based virtual hosts are impossible with HTTPS [1]. I've got two sites currently on different servers. Each is distinguished by a path prefix ("/a" and "/b"), so they aren't dependent on the domain name and can be installed in the same virtual host. The boss wants them consolidated on one server, and to plan for additional sites in the future. The problem is the certificates. A certificate is domain-specific, and it looks like you can have only one per virtual host.

So person A types https://a.example.com/a/ and it authenticates fine, but person B types https://b.example.com/b/ and gets a "domain does not match certificate" dialog. (I have seen this in some cases, but haven't gotten it in my tests. But it may be because we're still using unofficial certificates and getting the "unknown certificate authority" dialog instead.) The only solutions seem to be using a general domain for all the sites, getting a separate IP for each one, or running them on nonstandard ports.

[1] http://www.modssl.org/docs/2.8/ssl_faq.html ("Why can't I use SSL with name-based/non-IP-based virtual hosts?")

[Jay] Correct. You can't have more than one SSL server per IP address, because the certs are IP based, not domain name based.

They have to be, if you think about it, because you can't spoof IP [1] the way you can spoof DNS.

[1] unless you manage a backbone.

[Brian] I think, if your example is true, then [IIRC, you'll have to do more research] you can spend the bucks to get a wildcard cert that will handle [a-g].example.com/blah just fine. Alternatively, get extra IP addresses, alias the eth as needed, and multiple single-host certs can be applied. That works just fine. A separate set of SSL stanzas in each virtual host section, virtual host by number, not by name.

You may, in that case, actually want to run a separate invocation of apache for the SSL side of things, so that you can do IP-based virtual hosts for SSL, and name-based virtual hosts for port 80.

[Ramon] Because encryption is set up before any HTTP headers are sent, name based vhosting with multiple certificates is not possible.

The only thing that does work is multiple vhosts with one certificate that validates all of them. I've done that successfully with a project vhost server on ssl for multiple software development projects. You can get a wildcard certificate from rapidssl http://www.rapidssl.com for $199.

They're a dirt cheap certificate provider BTW $69 for a two year standard webserver certificate accepted in most (if not all) browsers

If it were a small organization that would be a possibility. But we're part of a large organization and can't monopolize the entire domain (*.example.com). At the same time the sites are for multiple departments, and we haven't been able to come up with a *.subdomain.example.com that would satisfy all of them.

Oh wait, you're talking about wildcard IPs rather than wildcard domains? (checking rapidssl website) No, it is domains.

Hmm, getting a wildcard certificate would obviate the need for multiple certificates but that's actually the lesser of our problems. The greater problem is getting more IPs, justifying them, and putting in a new subnet for them. But I guess I'll tell management that if they really want these multiple domains on one computer, they'll have to factor a new block of IPs into the price.

Has anybody had experience with https://cert.startcom.org/ ? It appears to be a nonprofit project geared toward free certificates.

"The StartCom Certification Authority is currently undergoing an initial self and successive third party audit as required by various software vendors, such as Microsoft and Mozilla. This will lead to the natural support of the StartCom CA by the most popular browser and mail clients. Right now you still have to import our CA certificate into your browser, but chances are, that, during the life-time of your certificate (one year), your certificate will be supported without the need of the CA import."

Probably not an option for us yet, but it looks worth watching.

Duh, our netadmin pointed out that when the second site is moved over, we can take the IP from that computer. And my other site will replace seven other servers so we can take their IPs too. That'll last us past the foreseeable future. Anybody got a few HTTPS sites they need hosting for a year or two? (Just kidding.)

Mozilla hogging the screen

Neil Youngman (ny from youngman.org.uk)
Answered By Ben Okopnik

Mozilla has started hogging my screen. I can select other windows, but if Mozilla is maximised it remains in front of them. There is presumably a setting somewhere that is causing this behaviour, but the only setting I can find I can't seem to change. FYI, this is in KDE.

If I right click the Mozilla title bar and select advanced->special window settings->preferences, there is a checkbox either side of the "keep above" setting. The checkbox on the right is checked and greyed out. With a little fiddling I can get it unchecked, but if I click OK and then reopen the window to check it, I find that it is selected again.

I don't know if that setting is the source of the problem, but the other windows don't have it checked, so it's a good candidate.

Any ideas how to fix this one?

OK. Going down into the "special window settings" wasn't necessary. If I just use "advanced->keep above others" it toggles that checkbox. It's annoying and a little confusing that it can't be changed from "special window settings".

[Ben] Hmm. Perhaps one or two - my Firefox started doing some ugly thing a while back, so I whacked it over the head a couple of times, and will happily relate what LART I used. Mind you, this is in the nature of shotgunning rather than troubleshooting (I can hear the sounds of retching from the other techies here, but, hey, it works - and I didn't feel like pulling down a hundred meg or so of code and wanking through it.)

Move your ~/.mozilla to, say, /tmp/DOTmoz.
Start Mozilla.
If $UGLY_BEHAVIOR is still present, uninstall the mozilla package (making sure to blow away, or at least _move_ away all the stuff in "/usr/lib" and "/etc") and reinstall from scratch. If it's still there, curse life and file a bug. Otherwise -
Make a copy of your new ~/.mozilla (as, say, /tmp/DOTmoz_default.) Start replacing the subdirectories in the one in $HOME, one at a time, from /tmp/DOTmoz until the problem reappears. Narrow it down to the specific file, then diff that file against the default one. The line causing the problem should be relatively obvious - since Mozilla uses more-or-less sensible, descriptive names for their config variables.

To (mis)quote the folks at the Mozilla Project, "it worked for me."

I'd say this was starting from the wrong end. Possibly my fault because I flagged it as Mozilla hogging the screen. With window behaviours like this, it's far more likely to be a window manager issue.

I have solved the problem now. You should have seen a followup email on the list.

[Ben] I've had similar problems (back in Netscape days, actually), and thought that it was the WM originally - it just made sense. Turned out to be that Netscape was doing some of its own craziness, at least in that case; I can definitely see where it could just as easily be the WM.

Couple of perl related questions

Suramya Tomar (security from suramya.com)
Answered By Ben Okopnik, Jimmy O'Regan

Hi Everyone, I have a couple of questions for the perl experts that seem to lurk around the TAG mailing list.

[Ben] Never heard of any around here. However, I do play one on a center stage once in a while, so I'll try to help.

I was playing around with the Yahoo Search API and decided to write a program that uses it to search for images based on user input and creates a collage from the results. I actually managed to get it to work (http://scripts.suramya.com/CollageGenerator) but need some help in fine tuning it.

The program consists of two parts: the frontend which is a php page and the backend which is a perl script. The PHP frontend writes the user input to a mysql DB which another perl script I call wrapper.pl checks frequently, when it finds a new row it calls the collage.pl that creates the collage.

[Jimmy] Um... is there any reason why the information has to be in a database? It seems like you're over complicating things: PHP is able to download files (IIRC, fopen can open from URLs), and Perl is well able to do CGI (use CGI , and can be embedded in HTML like PHP using HTML::Embperl (http://search.cpan.org/~grichter/HTML-Embperl-1.3.6/Embperl.pod). This page (http://www.cs.wcupa.edu/~rkline/perl2php) has a Perl to PHP 'translation', but it's also good for the other direction.

You can also directly embed Perl in PHP (http://www.zend.com/php5/articles/php5-perl.php), and PHP in Perl (http://search.cpan.org/~karasik/PHP-0.09/PHP.pm http://search.cpan.org/~gschloss/PHP-Interpreter-1.0/lib/PHP/Interpreter.pm), and Perl can read some PHP directly (http://search.cpan.org/~esummers/PHP-Include-0.2/lib/PHP/Include.pm).

The original machine where my site was hosted was not a very powerful machine so the collage creation took ages.

So I decided to use a client server model where I could run the backend on multiple machines and have each of them process a small portion of the requests which the system got. Thats why there's a DB involved so that I can keep track of who's working on what query and the backend can run on my home machine or a different more powerful system.

Right now I am running just one backend process but once I get most of the bugs worked out I will prob put them on other systems I have. (Just to decrease the wait time..)

Thanks for the links though, They will be useful in other programs I am thinking about.

Now my first problem is that I am using the following function to download the images to the local system for processing and I am not comfortable with it.:

sub download_images
{
  my $url = shift;

  $url =~ s/\"/\%22/g;
  $url =~ s/\&/\%26/g;
  $url =~ s/\'/\%27/g;
  $url =~ s/\(/\%28/g;
  $url =~ s/\)/\%29/g;
  $url =~ s/\*/\%2A/g;
  $url =~ s/\+/\%2B/g;
  $url =~ s/\;/\%3B/g;
  $url =~ s/\[/\%5B/g;
  $url =~ s/\]/\%5D/g;
  $url =~ s/\`/\%60/g;
  $url =~ s/\{/\%7B/g;
  $url =~ s/\}/\%7D/g;
  $url =~ s/\|/\%7c/g;


  # print "Getting " . $url . "\n";

`wget -T 1 -t 1 -q $url`;
}

Is there a way I can download the images fast to my computer without having to use wget? I download upto 10 images eachtime for creating a collage. I don't like passing results I get from the net directly to a shell but this is the only way I could get it to work. Another disadvantage of wget is that if it can't download an image it takes forever to timeout and goto the next url in the list.

[Ben] Take a look at the LWP toolkit at http://cpan.org ; it contains support for any kind of HTTP/FTP/NNTP/etc. usage you might want from within Perl. The above can be done this way:

use LWP::UserAgent;
use HTTP::Request;

# Create user agent
my $u = LWP::UserAgent -> new;

# Create request
my $r = HTTP::Request -> new( GET => "$url" );
# Configure the request however you want - e.g.,
$r -> timeout( 10 );

# Pass request to UA
my $ret = $u -> request( $r );

print "Error fetching $url" if $ret -> is_error();

There are much simpler ways to do it - i.e.,

perl -MLWP::Simple -we 'mirror "http://foo.bar.com"'

does the whole thing in one shot - but it's not nearly as flexible as the above approach, which allows tweaking any part of the interaction.

Thanks for the info. I will check out this package. It looks like it does what I want. How is this package speed wise/resource usage wise?

[Ben] Forgot to mention: this is untested code, just off the top of my head - but stands a reasonably good chance of working. See 'perldoc LWP::UserAgent' and 'perldoc HTTP::Request' for the exact public interface/usage info.

Ha ha, don't worry I had guessed that this was the case. Afterall I can't expect you to do all the work for me... I will try out the code and let you know how it went.

The second problem is that my mysql connection seems to drop at random times during execution. What can I do to prevent the mysql server from going away?

[Ben] 1) Stop shelling out. If in doubt, read "perldoc perlsec" (Perl security considerations) - and then stop shelling out. This includes command substitution (backticks) as well as the 'system' call.

2) In any interaction involving file system calls the timing of which could affect the Perl functions, force the buffers to autoflush by setting the '$|' variable to non-zero. Oh, yeah - and stop shelling out.

Below is the code I use in wrapper.pl to check the DB for changes:

See attached wrapper.pl.txt

The script usually dies around the last $update->execute. What I think might be happening is that the collage.pl is taking too long to run and the DB connection times out, is that possible? Can I force the connection to not timeout? (I did try searching on google but didn't find any ways of changing the keep connection alive variable from a script).

Any idea's/suggestions? Thanks in advance for the help.

PS: Any suggestions on improving the script would be most welcome as I am using this to try to learn perl.

Securing rsync

Mike Orr (LG Contributing Editor)
Answered By Benjamin Donnachie, Kapil Hari Paranjape

I'm trying to get rsync access to an OS X server with a paranoid sysadmin who doesn't know much about Unix progams. (He's a GUI kind of guy.) He's offered me FTP access to one directory but I'd really like to use rsync due to its low-bandwidth nature and auto-delete feature (delete any file at the destination that's been deleted at the source). His main desire is not to grant a general-purpose account on the server, so if I can convince him that rsync+ssh can be configured to grant access only for rsync in that directory, I may have a chance. But since they're two separate programs (as opposed to *ftpd and mysqld, which can have private password lists for only their program), I'm not sure how to enforce that. Would I have to use rsyncd alone, which I guess means no encryption? (Granted, ftp has no encryption either, but I think he's just using that due to lack of knowledge of alternatives.)

(And when is ssync going to arrive, to avoid this dual-program problem?)

[Benjamin] Take a look at rssh (http://www.pizzashack.org/rssh/index.shtml) or scponly (http://sublimation.org/scponly) - both can be used together with ssh to restrict access to just rsync.

However, access to a single directory would probably require a user jail - - all is explained in the rssh and scponly docs, but it's not really for your "GUI" types.

[Kapil] I suppose you mean something that combines ssh and rsync. In any case your particular problem might be solved by means of an authorized_keys file entry that looks like (this is all in one line one line)

from="202.41.95.13",command="rsync -aCz --server --sender $SRCDIR  .",
no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty
ssh-dss
AAAAB3NzaC1kc3MAAACBAMzBqJtkx52y3IL9g/B0zAna3WVP6fXDO+YZSrb8GsZ2Novx+1sa
X/wBYrIXiKlm0LayvJpz7cF17ycWEyzANF9EivYbwMXPQWecxao82999SjKiM7nX2BUmoePN
iUkqpnZuilBS2fPadkTZ7CSGevJ2Y9ryb1LOkFWkdBe2c4ETAAAAFQCUk+rB5HRYj0+KIn5H
fiOF0dQtvwAAAIA7ezUaP6CpZ45FOJLunipNdkC0oWj3C5WgguwiAVEz3/I5ALAQ9hdmy+7+
Bi0hUGdkTcRoe4UPMgsahdXLZRNettMv+zdJiQqIiEnmzlQxlNY2LBlrfQyRwVU1SW3QWGog
tssIoeOp9GRx7N5H2ZAzMoyGBaUDfHVQueI2BeJiGwAAAIEAhV0XWIm8c2hLAGRyYZeqi5G3
yDAB0DZPyiuYyfnBav3nqjkO65faDuajKUv6as90CGYVlixWO+tjOL+/D9c0DoqdMllPEiZw
aGBxd96o1pVrsFw/Ff0jJOtxzj+/Tzgaw9AdI0Usgw1cfXwWS1kJlhXqR00O/ow/XATejWpW
0i8= kapil@neem

Here you must put the appropriate source directory in $SRCDIR.

The authorized key file can be put in a dummy users directory. This dummy user should have appropriate read/write permissions for the directory in question.

As an alternative you can use a configuration file "--config=$FILE" in place of $SRCDIR.

Once this is done, the owner of the SSH private key associated with the public-key (which is the bit that starts ssh-dss AAA....) can connect to the ssh server and start the above command and only the above command.

Boot from cd via floppy

Francis Daly (francis from daoine.org)
Answered By Thomas Adam, Ben Okopnik

Hi there,

I'm not a TAG subscriber, so I can't see the list archives to verify, but hopefully this mail isn't repeating something that you've already had a dozen times this month.

[Thomas] So far, you're the first.

From September's gazette: "my machine only boots from floppy, and I want it to boot from cd" might be addressed with a smart boot manager, such as sbm. The debian (sarge) one credits James Su and Lonius as authors, and says it was downloaded from http://www.gnuchina.org/~suzhe , but it looks like the useful content can now be found at http://btmgr.sourceforge.net

[Thomas] Indeed. It has been mentioned in the LG in the past (twice by me, and once by Ben, I believe.)

[Ben] Wasn't me; I hadn't run across SBM until now.

[Thomas] It's OK, and provides a lot of elaborate features that can be quite interesting on certain types of hardware, it has to be said.

[Ben] As is often the case, Debian already has it as a package (pretty amazing folks, those Debian maintainers!) -

ben@Fenrir:~$ apt-cache search smart boot
bmconf - The installer and configurator of the Smart Boot Manager
sbm - Smart Boot Manager (SBM) is a full-featured boot manager

As Francis has already mentioned, though, it won't boot USB devices. Too bad; that would make it quite useful, especially given that modern kernels are too big to fit on a floppy anymore.

By the way - the fact that they are too big annoys the hell out of me. There are plenty of folks out there who need floppy-based booting - troubleshooting and booting weird hardware configurations are two situations where that capability can be critical - and "new systems all come with a CD-ROM" is NOT equivalent to "all existing systems have a CD-ROM". Yeah, older kernels, whatever; as time goes on, those become less and less useful - and support less and less common hardware. I'll admit that I'm coming from ignorance here, but - there should have been a way to make the kernel modular enough to provide the "compile small kernel" option instead of just losing this important capability.

Thanks for the reply. Oops -- I hadn't spotted that. I did try searching for "sbm", and all I found was a (presumably) mis-spelled samba config file. But now that I try again, searching for "smart boot manager", I see that it does appear in the archives.

No harm done.

"sbminst" it to a floppy to confirm that it can use your hardware, then consider putting it in your primary disk mbr, consigning lilo or other boot loader to a partition or secondary disk. Of course this last bit presumes that "my machine only boots from floppy" really means "my machine only boots from floppy or one hard disk", but that's probably a reasonable assumption.

Worked for me with an ATAPI cd drive that the BIOS didn't like. I suspect it won't work with the SCSI cd in the original problem, sadly. And am almost certain that it also won't work with the USB stick in the original original problem. So it isn't a full solution -- or even a useful solution in these specific cases -- but it might help someone with a slightly different problem.

Security implications of root login over SSH

Riza Aziz (telefonixar from yahoo.com)
Answered By Kapil Hari Paranjape

Hi there,

I'm wondering if it's wise to allow a remote user within the LAN to log in as root, by adding that user's public key to root's "authorized_keys" for that machine.

[Kapil] There is an "sudo"-like mechanism within SSH for doing this. In the authorized_keys file you put a "command=...." entry which ensures that this key can only be used to run that specific command.

All the usual warnings a la "sudo" apply regarding what commands should be allowed. It is generally a good idea to also prevent the agent forwarding, X11 forwarding and pty allocation.

Here is an entry that I use for "rsync" access. (I have wrapped the line and AAAA.... is the ssh key which has been truncated).

from="172.16.1.28",command="rsync -aCz --server --sender . .",
no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty
ssh-dss AAAA..... rsyncuser

This page edited and maintained by the Editors of Linux Gazette
HTML script maintained by Heather Stern of Starshine Technical Services, http://www.starshine.org/

Published in Issue 119 of Linux Gazette, October 2005

The Answer Gang

Linux Gazette 119: The Answer Gang (TWDT) The Answer Gang 119:

...making Linux just a little more fun! (?) The Answer Gang (!)
By Jim Dennis, Jason Creighton, Chris G, Karl-Heinz, and... (meet the Gang) ... the Editors of Linux Gazette... and You!

We have guidelines for asking and answering questions. Linux questions only, please.
We make no guarantees about answers, but you can be anonymous on request.
See also: The Answer Gang's Knowledge Base and the LG Search Engine

Converting bitmap to binary

From Adam Engel

Answered By: Ben Okopnik, Jimmy O'Regan

Hey Gang,

Does anyone know of a script or program that can convert a bitmap image to binary code and print the line(s) of ones and zeros to standard output?

[Ben] Adam, if I didn't know you, my "homework detector" sense would be tingling. As it is, I'm still a bit boggled as to why you'd want to do such a thing, but - all the standard tools that I can think of, off the top of my head, do octal and hex. Binary, well... heck, I'd do what I always do when it takes me more than a few seconds to think of an answer to that kind of question: reach for Perl.

perl -wne'printf "%08b ", ord $_ for split //' foobar.bmp

That'll give you a line of space-separated, 8-bit binary numbers representing the ASCII value of each character in the file. Just to play with the idea, converting it back wouldn't be any harder:

perl -wne'print chr eval "0b$_" for split' foobar.binary

[Jimmy] Sounds like pbm (http://netpbm.sourceforge.net/doc/pbm.html), minus the header:

P1
# feep.pbm
24 7
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 1 1 1 1 0 0 1 1 1 1 0 0 1 1 1 1 0 0 1 1 1 1 0
0 1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 1 0
0 1 1 1 0 0 0 1 1 1 0 0 0 1 1 1 0 0 0 1 1 1 1 0
0 1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0
0 1 0 0 0 0 0 1 1 1 1 0 0 1 1 1 1 0 0 1 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

The netbpm tools are a standard part of Linux distributions everywhere. Image Magick and The Gimp are able to write it too.

[Ben] I just tried converting some text to PBM format (using 'convert' from ImageMagick), and it's not doing anything like the above; in fact, it created a file that's mostly full of nulls - which displays a white page with the word "hello" in the center when looked at with an image viewer.

Looking at a 1k chunk at the top of it, I get the following ("^@" is how 'less' displays a null):

ben at Fenrir:~$ printf "hello"| convert text:- pbm:-|head -c 1k|less
P4
612 792
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
^@^@^@^@^@^@^@^@

[Jimmy] Whoops. I meant the 'Plain PBM' format, which you get using the pnmtoplainpbm program.

[Ben] [grin] One more time, with gusto...

I think you mean "pnmtoplainpnm" (I checked the Debian file list as well as doing a Google search - which found nothing but rebuked me with "Did you mean: pnmtoplainpnm"; there's no 'pnmtoplainpbm' in sight.) That, however, doesn't seem to do it either:

ben at Fenrir:~/Pics$ pnmtoplainpnm smile.pnm | head -1k| less
P3
15 15
255
191 191 191  191 191 191  191 191 191  191 191 191  191 191 191  0 0 0
0 0 0  0 0 0  0 0 0  0 0 0  191 191 191  191 191 191
191 191 191  191 191 191  191 191 191
191 191 191  191 191 191  191 191 191  0 0 0  0 0 0  255 255 0
255 255 0  255 255 0  255 255 0  255 255 0  0 0 0  0 0 0
191 191 191  191 191 191  191 191 191
191 191 191  191 191 191  0 0 0  255 255 0  255 255 0  255 255 0
255 255 0  255 255 0  255 255 0  255 255 0  255 255 0  255 255 0
0 0 0  191 191 191  191 191 191
191 191 191  0 0 0  255 255 0  255 255 0  255 255 0  255 255 0
255 255 0  255 255 0  255 255 0  255 255 0  255 255 0  255 255 0
255 255 0  0 0 0  191 191 191
191 191 191  0 0 0  255 255 0  255 255 0  0 0 0  0 0 0
255 255 0  255 255 0  255 255 0  0 0 0  0 0 0  255 255 0
255 255 0  0 0 0  191 191 191
0 0 0  255 255 0  255 255 0  255 255 0  0 0 0  0 0 0
255 255 0  255 255 0  255 255 0  0 0 0  0 0 0  255 255 0
255 255 0  255 255 0  0 0 0
0 0 0  255 255 0  255 255 0  255 255 0  255 255 0  255 255 0
255 255 0  255 255 0  255 255

According to the NetPBM project page/"pbm" man page, "Plain PBM" is created by the "pnmtoplainpnm" utility - although they do state that it only works with monochrome images.

Ah-HA! I've got it - sorta. "pnmtoplainpnm" takes either a PNM or a PBM file, and applies a /reductio ad absurdum/ algorithm to produce the "simplest" version of the input format is. This does indeed do... something similar to what the manpage described:

ben at Fenrir:~/Pics$ printf "foo"|convert -crop 25x15+40+45 text:- pbm:-|pnmtoplainpnm
P1
25 15
0000000000000000000000000
0000000000000000000000000
0000110000000000000000000
0000100000000000000000000
0001110111100011110000000
0000101100110110011000000
0000101000010100001000000
0000101000010100001000000
0000101100110110011000000
0000100111100011110000000
0000000000000000000000000
0000000000000000000000000
0000000000000000000000000
0000000000000000000000000
0000000000000000000000000

I don't know that it actually matches the question that was asked, though. :)

[Raj] Not sure if it might help you, but you can have a look at aalib ASCII-art library http://aa-project.sourceforge.net/aalib . It converts jpegs to ascii.

Or do you want to see the raw binary as it is stored in the bmp file ?

[Ben] Well, sorta. Very sorta. Unless you're a C programmer with time enough to write a converter that uses aa-lib. I've always thought that their demo ('bb') was fantastic, but... it's not something that ever really caught on, and I'm a sort of a Luddite in Linux clothing, so my opinion doesn't count.

What you might be thinking about is "aview/asciiview" - which is what comes closest to "converting" JPG to ASCII; the latter displays JPGs, as well as any other format recognized by the NetPBM kit, in an extremely rough ASCII approximation. However, "aview", which can only read PNM-formatted images, is capable of surprising image quality on a terminal with a tiny font:

xterm -fn 5x7 -geometry 1000x1000
# Enter this command in the new xterm
convert logo: pnm:-|aview -driver slang

All good fun, but - still doesn't answer the question as posed. Unless I've totally misread it.

Actually, I just recently downloaded NetPBM for another application for JPEG conversion. As far as the "GUI" tools like Imagemagick and The Gimp go, I being a gui-phobe and graphically challenged to boot, I rarely use them except to demonstrate that there is "another way" to people who have thrown away several hundred dollars a year on Adobe Photoshop. There is no "practical" use for my question. It came out of my recent study of "sed" and the movie "Charlie and the Chocolate Factory" (in which Johnny Depp transforms a giant chocolate bar into a "TV" -sized package with an imaginary contraption). The "real" exorcize,

[Jimmy] Come come. It doesn't sound particularly difficult, let alone demonic.

in Chapter Six of O'Reilly's "Sed & Awk" concerns a bit-mapped file that is represented in binary format. Beyond the scope of the exorcize/sed script, I've been trying to "translate" bitmap images to binary code and vice versa to see if -- I know, I know, this is STUPID, but I never pretended to be a Guru, just a GoTTO -- one can send images as text-files to be "rebuilt" upon receipt so people with low-bandwidth connections wouldn't have to wait forever to download images.

[Jimmy] Erm... I think you have things backwards here. Getting images in a binary format takes less bandwidth than getting the same thing encoded as text.

I chose "binary" merely because that was the code used in the exorcise. My excuse for tardiness handing in my homework is part medical -- had to spend a few days off-line in every sense of the word, and part rational: I finally asked myself, "wouldn't people have thought of this before?

[Jimmy] Yep. uuencode, base64, etc. etc.

If it makes you feel any better, it is possible to use a text representation of binary data with the data: URI (http://www.ietf.org/rfc/rfc2397), which Mozilla and Opera support.

Here's a simple example: data:text/plain;charset=utf8,Ksi%C4%85%C5%BCka%20kucharska That just gives you a simple text page with some Polish text.

Here's a bigger example:

See attached data-test.html

Is that what you were thinking about?

That was it exactly. All that hullaballoo for "Dan and James' going-away present." But thanks. After ten years of sending images back and forth, it just occurred to me that I had no idea what was under the hood. Hence, it made "sense" that plain text would be "lighter" than gifs, jpegs, xml images etc. Thanks again for the explanation and the demo.

[Jimmy] Heh. I already had that example made up: it made sense to the original recipient :)

Hence, it must be doable and I merely can't figure out how to do it, or it's doable but not worth the time/effort and was a silly idea in the first place." On the other hand, if the numbers can be deconstructed in Perl, and reconstructed, why not? They laughed at Willy wonka, and look at him NOW.

freeshell.org service outage

From Rick Moen

Answered By: Jay R. Ashworth, Ben Okopnik, Sindi Keesan

I've been paying a little closer attention to SMTP errors, since the migration of LG's public mailing lists. Here's one for TAG subscriber Sindi Keesan. (I just received one each of these following Deepak and Ben's posts to the "TFTP problem" thread, and undoubtedly will get another for this one.)

  From MAILER-DAEMON Thu Jun 02 11:19:23 2005
  From: Mail Delivery System <Mailer-Daemon@linuxmafia.com>
  To: tag-bounces@lists.linuxgazette.net
  Subject: Mail delivery failed: returning message to sender
  Date: Thu, 02 Jun 2005 11:19:22 -0700

  This message was created automatically by mail delivery software.

  A message that you sent could not be delivered to one or more of its
  recipients. This is a permanent error. The following address(es) failed:

    keesan@freeshell.org
      Unrouteable address

    [goes on to provide a copy of the undeliverable list post]

A lot of us have become accustomed to calling these "bounces" and disregarding them because they're so often cryptic and impenetrable. (My SMTP server, in general , gives pretty clear diagnostic messages, and yet this one was obscure to me, too.) Sometimes, the pedants among us distinguish Delivery Status Notifications (DSNs) from "bounces", where the former are three-digit SMTP-standard error codes and matching explanatory text, generated by the remote SMTP host (MTA process) during an SMTP conversation.

In this case, there's none of that "550 User unknown" or similar DSN stuff, and I was left curious what "Unroutable address" means, here -- especially since Ben and others have a pretty high opinion of Stephen M. Jones's "SDF Public Access UNIX System, INC." operation at freeshell.{org|net}.

I was intending to attempt a manual SMTP session with that system (by telneting into its mail exchanger (MX). The first step, then, is to ask the public DNS where freeshell.org's MXes are:

  [rick@linuxmafia]
  ~ $ dig -t mx freeshell.org +short

  ; <<>> DiG 9.2.4 <<>> -t mx freeshell.org +short
  ;; global options:  printcmd
  ;; connection timed out; no servers could be reached
  [rick@linuxmafia]

Hmm. Can that be right? No nameservers can be reached that are authoritative for the domain? First, let's cross-check to make sure I'm getting meaningful results for similar queries on other domains (i.e., that I don't just have network or DNS-access problems of my own):

  [rick@linuxmafia]
  ~ $ dig -t mx apple.com +short
  30 eg-mail-in1.apple.com.
  10 mail-in3.apple.com.
  10 mail-in4.apple.com.
  10 mail-in5.apple.com.
  [rick@linuxmafia]
  ~ $ dig -t mx linuxmafia.com +short
  10 linuxmafia.com.
  [rick@linuxmafia]
  ~ $

Yep, that's all looking good. Let's see what IPs are listed as freeshell.org's authoritative nameservers in the whois servers:

See attached whois-output.txt

Er, I might be missing something, but having all of one's nameservers be in-domain seems like a bit of a hazard. Sure, the top-level nameservers will also have their IPs as part of the DNS's "glue records", but the rest of us won't. And having only two nameservers is a bit thin.

[Ben] Indeed, it is a hazard; the few times that SDF has gone down, it was like being shifted sideway into an alternate universe in which it had never existed. The response from web browser, fetchmail, etc. amounted to "Freeshell? What's a Freeshell? Go away, you silly man - we have no time for psychotics with an overactive imagination."

As I've found out while researching my response to Jay, I think we've now found part of the reason for Stephen's problem: His third nameserver is getting ignored (not used), because of obsolete glue records in his parent zone. He needs to fix that.

It would be nice if Sindi Keesan or someone else whose domain name doesn't have "linux" in it would advise Stephen of that, and gently lead him by the hand to the www.dnsreport.com test CGI -- as that gives a nice overview of his problems (and, basically, a checklist).

[Sindi] I would be happy to send along a message to him from my address here if you tell me exactly what to say and where to address it to.

This reminds me of when someone knowledgeable at my local bbs figured out why the electric company's online billing site went in little circles, but they did not want to hear about it. (Nor were they interested in the fact that Spamassassin was dumping their enormous emailed bills for five different reasons, including green fonts, too many images, odd looking subject line or from, too much HTML, and 'porn', and the mails were too large to receive at my address without Spamassassin).

OK, try this:

According to the report on http://www.dnsreport.com/tools/dnsreport.ch?domain=freeshell.org , your third nameserver (ns-c.freeshell.org) isn't in the authoritative list in the .org records, even if you have it in the zonefile. Because of that, it probably won't get DNS queries about freeshell.org, which may partially explain the outage we had recently.

Also (as mentioned in that report), "freeshell.org." in your zonefile's SOA record is wrong, and probably should be "ns-a.freeshell.org."

That report also make some sensible-sounding suggestions about timeouts to tweak in the SOA record, which you might consider.

[Sindi] Who do I send this to? I am not very familiar with sdf, just use it for email and website.

Re-checking my first post to this thread:

See attached whois-output.txt

The indicated e-mail address appears to be that of Stephen M. Jones, proprietor.

Suggestion: "whois" is your friend.

[Sindi] I sent it to the address below with a short preface stating that a 'friend' suggested I pass along this information. Thanks. This situation reminds me of that of a friend whose daughter will not accept email from him if it is properly spelled - she knows he is dyslexic and insists that he write it himself, so when I write it for him we have to send it from his address not mine, and make sure to introduce some spelling errors. (And then her 8 year old mysteriously sends perfectly spelled emails 'all by himself'.)

[Jay] Two nameservers is indeed a bit thin... but on the other point, unless my understand of DNS is also thin, the parent nameserver is always going to hand you the glue, is it not?

It's going to hand you the glue records if it has them. One of the reasons I like the "DNS Report" test at http://www.dnsreport.com is that it shows you, by implication, the immense variety of ways to screw up one's DNS -- and one of them is to have missing or incorrect glue records in the parent zone. Recommended facility, anyway.

[Jay] And in return, nice tip. :-)

Very cool site. I wonder if he has a version that returns something more easily parseable, by, say, Nagios. Or, alternatively, will make his script available. Must look closer.

In fact, if you look closely at http://www.dnsreport.com/tools/dnsreport.ch?domain=freeshell.org , it is evident that Stephen M. Jones did at some point deploy a third nameserver, but that it's missing from the parent records, which explains some of his fragility problems. (He also has some minor SOA errors.)

[Jay] I was a touch surprised, though that you didn't demonstrate the handy-dandy "+trace+ option to dig, which I fell in love with the minute I found it:

Neat. Here's the result for linuxgazette.net:

See attached dig-trace-output.txt

I haven't done nearly enough playing around with new DNS tools: I'm one of those codgers who've been hanging onto nslookup and sulking about its ongoing demise. Thank you for pointing out that trick!

[Jay] I'm trying to figure out a reasonable way to automate running it and looking for changes; it's not quite tuned for that. Perhaps the dnsreport code would be easier to use that way.

[Jay] It automatically traces the domain down from the root, showing you the salient information at each step of the way; the important bit in this case was:

> freeshell.net.          172800  IN      NS      ns-a.freeshell.org.
> freeshell.net.          172800  IN      NS      ns-b.freeshell.org.
> ;; Received 82 bytes from 192.12.94.30#53(E.GTLD-SERVERS.net) in 81 ms
>
> ;; reply from unexpected source: 65.32.1.80#53, expected
> 192.94.73.20#53
> ;; Warning: ID mismatch: expected ID 31090, got 36862
> ;; reply from unexpected source: 65.32.1.80#53, expected
> 192.94.73.20#53
> ;; Warning: ID mismatch: expected ID 31090, got 36862
> ;; Received 31 bytes from 192.67.63.37#53(ns-b.freeshell.org) in 45 ms

Note that a) it thinks those servers are in freeshell.org, not .net, and b) that it appears that neither of them are answering the phone.

You can see that it did get an answer, though I'm a touch irked at dig that it didn't tell us what that answer was. The 65.32 servers are the customer resolver servers for Road Runner TampaBay, which is my uplink; why it saw fit to answer for itself I don't know -- clearly, since I did this from a Linux box, it should not have even been being asked...

But it appears still not to be running; perhaps the gent gave up?

Looks like he had a one-day outage, and is back.

[Jay] Well, good. We don't need that sort of service much anymore, but those that need it... need it.

Here's the list from my own domain:

   Domain servers in listed order:
      NS1.LINUXMAFIA.COM   198.144.195.186
      NS.PRIMATE.NET   198.144.194.12
      NS1.VASOFTWARE.COM   12.152.184.135
      NS.ON.PRIMATE.NET   207.44.185.143
      NS1.THECOOP.NET   216.218.255.165

For some reason, my Tucows / OpenSRS registration lists the authoritative nameservers' IP addresses in the public DNS, while Stephen M. Jones's doesn't. I'm not clear on why this is.

Anyhow, that's at least something to go on. Let's find out what IP addresses the authoritative nameservers have:

  [rick@linuxmafia]
  ~ $ host NS-A.FREESHELL.ORG
  NS-A.FREESHELL.ORG has address 192.94.73.20
  [rick@linuxmafia]
  ~ $ host NS-B.FREESHELL.ORG
  NS-B.FREESHELL.ORG has address 192.67.63.37
  [rick@linuxmafia]

Well, at least that much of his DNS is working.

[Ben] Wouldn't that be your DNS that's working? Unless I'm mistaken, "host" uses your /etc/resolv.conf to look up hosts - unless you specify another DNS server explicitly.

Well, DNS being the distributed system that it is, you're always using some client piece and some server piece. But what I meant is that at least that much of his DNS information is working (accessible and useful). It might very well have been cached in my or some other non-authoritative nameserver's records, yes. But I was wanting to fetch his authoritative nameservers' IPs from somewhere -- anywhere -- so that I could ask them questions directly, as the next step. Nothing like getting DNS answers straight from the horse's mouth, if you don't mind the rather unsanitary metaphor.

Let's ask the nameservers explicitly by their IP addresses, to make double-sure the query's going to the right place:

  ~ $ dig -t mx freeshell.org @192.94.73.20 +short

  ; <<>> DiG 9.2.4 <<>> -t mx freeshell.org @192.94.73.20 +short
  ;; global options:  printcmd
  ;; connection timed out; no servers could be reached
  [rick@linuxmafia]
  ~ $ dig -t mx freeshell.org @192.67.63.37 +short
  [rick@linuxmafia]
  ~ $

How odd. Looks to me like the first nameserver doesn't respond, and the second returns some sort of null result. Just out of old-fogydom, and as a cross-check on "dig", let's do the same query using nslookup (a tool that's now deprecated, in general):

  [rick@linuxmafia]
  ~ $ nslookup -query=mx freeshell.org 192.94.73.20
  ;; connection timed out; no servers could be reached

  [rick@linuxmafia]
  ~ $ nslookup -query=mx freeshell.org 192.67.63.37
  Server:         192.67.63.37
  Address:        192.67.63.37#53

  ** server can't find freeshell.org: SERVFAIL

  [rick@linuxmafia]
  ~ $

[Ben] I believe that "host" is the recommended replacement for "nslookup" these days; I groused a bit about having to learn its syntax, but it's quite nice once you do. It's a sort of a cross between "dig" and "nslookup":

host -t mx freeshell.org 192.67.63.37
freeshell.org           MX      50 smtp.freeshell.org

Ah, it appears that I've underestimated the thing. Thanks.

In DNS lingo, SERVFAIL means that the domain does exist and that the root name servers have information on it, but that its authoritative name servers are not answering queries about it. So, basically Stephen M. Jones has one of his two nameservers offline, and the other misconfigured to the point that it stutters and faints when you ask it questions.

I hope it's a temporary glitch, but this (among other things) points out why continuity of DNS service is so important, and why two nameservers really aren't quite enough.

http://www.dnsreport.com/tools/dnsreport.ch?domain=freeshell.org is also interesting, giving an overview of just how much is broken here (a lot ) . (The freeshell.net variant of the domain has the same problem for the same reasons.)

Ben mentioned on a private mailing list that Stephen's a good guy and performs a generous service to the public but for reasons of personal experience loathes Linux. I vaguely remembered when that came about, and have re-found the rant he posted at the time, which still makes interesting reading:

http://web.archive.org/web/20010712145226/http://www.lonestar.org/sdf/

[Ben] Yeah, that was what I'd based my statement on; that, and the dance of boundless joy that he performed on the Freeshell list after the move was done (it's archived there, but it's not web-accessible AFAIK.)

I'll drop Stephen a brief, polite heads-up e-mail. I hope he won't mind my address. ;->

Stephen goes into a little more detail about his circa-2001 disenchantment with Linux, here:

http://mail-index.netbsd.org/port-alpha/2001/12/28/0008.html

But the guy I shave did some pretty thorough analysis at the time:: http://www.crackmonkey.org/pipermail/crackmonkey/2001q3/021695.html

Oh, what the heck, I might as well just quote it, because it's still relevant:

 From rick@linuxmafia.com Sun, 15 Jul 2001 13:59:04 -0700
 Date: Sun, 15 Jul 2001 13:59:04 -0700
 From: Rick Moen rick@linuxmafia.com
 Subject: [CrackMonkey] How come I am just hearing about this?

begin Bob Bernstein quotation:

> Found on the netbsd-advocacy list:
> http://www.lonestar.org/sdf/

It's got to really suck, being sysadmin of a public-access Unix system: You'll have an ungodly number of careless users, plus you have to worry about attacks both from arbitrary remote locations and attacks both by your users and by outsiders masquerading as legitimate users. When the day comes of you suddenly realising that your site has for some time been massively compromised, more often than not, you have only surmises about how entry and compromise occurred.

Numerous of Stephen Jones's statements suggest that such was the case with freeshell.org (aka "SDF"):

> I'm even thinking of just removing telnet/ftp/pop3 all together...

Plaintext-authentication network access to shell accounts: check.

> ...we might as well had our passwords in plain text as LINUX's use of
> encryption is about twice as good as Microsoft's.

Unless I misremember very badly how the login process works, passwords are not processed in kernelspace. Some Unixes introduce a PAM layer, while others do not. Some support MD5; others do not. But the irony is that Stephen's users did have their passwords in plaintext -- every time they did telnet/ftp/pop3!

In fact, it's obvious that Steven's brand new to security-mindedness:

> I've never felt security was important because I sincerely thought
> that a public system would be sacred ground to anyone be they a
> cracker or just normal user.

Poor bastard. I'd say he's had a rude awakening, except that I think it's not even begun, yet.

His July 11 note suggests that he was still running Linux kernel 2.2.18, which has been security-obsolete for a dog's age. (Note exception: Some distribution provide kernels with nominally earlier version numbers that have been patched to have the fixes introduced in nominally later kernels.)

Stephen writes:

> I blame LINUX due to the recent unveiling of the "oh, if root wasn't
                           ^^^^^^
> already easy to get, here is an easy way" bug in the execve() system
> call...

But that ptrace/execve() race condition (note: a local-user exploit) was not recent at all: It was a long while back. Wojciech Purczynski reported it to BugTraq on 2001-03-27:

http://www.securityfocus.com/templates/archive.pike?list=1&mid=171708&_ref=539250975

> ...where malicious code an be executed via almost any binary.
                                             ^^^^^^^^^^^^^^^^^

As Purczynski says, any SUID binary. But the point is that all this is very old news, 3+ months old. And extremely well known (as the "ptrace exploit").

Now, it seems likely that Stephen was still running an non-ptrace-patched 2.2.18 (or earlier) Linux kernel when the shit hit the fan, proving if nothing else did that he was asleep at the wheel -- but it's also clear that his system was security-exposed in a multitude of other ways, AND that he still is. (Example: He hasn't yet firmly deep-sixed telnet, POP3, and ftp inbound access mechanisms exposing shell passwords to the Net. He will.)

Let's do a taxonomy of root-compromise attacks (as opposed to DoS attacks and other categories): Rarely, these might be compromises of daemon processes or kernel network stacks from remote -- e.g., against vulnerable releases of BIND v. 8.x, lpr, or wu-ftpd. If the attack is not one of those, it must involve acquiring user-level access first, and then attacking the host from inside, impersonating a legitimate user. (In other words, the compromise of root authority is either from outside the host, or inside. Inside is much easier.)

The latter category breaks down further, according to how the attacker arranges to impersonate a legitimate user, into sniffing versus other. Sniffed passwords are, of course, what you get with standard deployments of telnet, non-anonymous ftp, and POP3 daemons -- and are a particularly ignominious way to get compromised. Stephen is only now thinking of shutting off this possibility -- so I fear he has other hard lessons yet to come.

The other ways of compromising shell-account passwords all trace back to the fact that users are pretty much always the weak element. If you let them, they'll use the same weak password everywhere. If you assign passwords yourself, change them at intervals, and remove the SUID bit from /usr/bin/passwd -- and sternly admonish the users not to expose their passwords through re-use on other systems -- they'll still do the latter, because they can, and because you can't stop them. Switch to one-time pad authentication, and they'll store the pads or seeds on vulnerable systems. And of course they'll ssh in, thinking that's unconditionally secure -- from compromised systems where attackers are logging all keyboard activity. And, remember, it takes only one user's shell access getting compromised. (Or, of course, the attacker might just sign up for a user account.)

You might be able to prevent system security from being shot in the foot by your users by requiring them all to use physical security dongles (e.g., SecureID) plus one-time pads. Maybe. But not on a public-access Unix system. In that sense, Stephen is screwed.

How so? Because he's doomed to having attackers occasionally get user-level access -- and protecting root against local users is much more difficult. While the remote attacker can fruitfully attack only running your network daemons and network stacks, as a local user he can attack any security-sensitive binary on your system -- a much wider field of targets. Stephen can try to keep installed software current, remove some, remove SUID/SGID from others, recompile using StackGuard, implement a capabilities model / ACLs, keep selected subtrees on write-protected media, and so on.

And he'll still get clobbered, from time to time. Odds are, he won't even be aware of compromise for quite a long time (as he probably wasn't, this time). Does he have IDSes set up? Of course not! He's "still against security". But that will change. Papa Darwin is a good, if ungentle, teacher.

NetBSD 1.5.1 on Alpha is going to be an eminently suitable system for him (even though the Alpha is doomed over the longer term). And Nick used to deliberately keep an Alpha on-line with a very vulnerable, antique OS load, just for the amusement value of watching x86-oriented kiddies' canned attacks crash and burn on it.

But Stephen has a longer-term problem, and it has nothing to do with kernel vulnerabilities -- let alone old ones that he should have long ago patched.

[Sindi] Thanks for pointing out to me that this thread has something to do with why sdf was gone yesterday, but it is way beyond my ability to understand. Could you summarize in a few sentences what this all means, for a beginning linux user with no computer training since Fortran IV?

As The Doctor says, "Ah, that takes me back -- or is it forward? That's the problem with time travel; you can never tell." ;->

(I likewise cut my teeth on FORTRAN, in my case playing around on university mainframes.)

If my post seemed a little bit meandering, it was because I was chasing down several things:

What on earth the cryptic SMTP error "Unrouteable address" means.
Why (rather unusually) that error wasn't accompanied by Delivery
What's going on with freeshell.org and its DNS.

The first question fascinated me because, of late, I've taken a particular interest in understanding SMTP-protocol mutterings -- and have improved my own machine's articulateness in that area -- and yet the quoted advisory ( issued by my machine's MTA) was about as clear as mud.

So, the answer to turned to be: "I, the lists.linuxgazette.net SMTP process, couldn't even attempt delivery at all, because the destination domain's DNS is completely non-functional.

That also furnishes the answer to question #2: There was no DSN conversation because my MTA couldn't even look up the detination IP, let alone talk to freeshell.org's SMTP host.

And answer #3 was: "It's unclear whether SDF's main services themselves went down, because SDF's nameservice outage made those unreachable by name, even if they were still running."

So, as I was saying, continuity of DNS service is really, really important to Internet services, and Stephen M. Jones's DNS for freeshell.org (as presently configured) has proven to be fragile. And thus your (recent) problem.

I sent a short, polite head-up advisory to Jones, about his DNS outage. He didn't respond, but I gather from your post that he must have fixed it!

[Sindi] Thanks for you explanation which I hope I understood.

I had Fortran in high school, back before our high school even owned a computer. We typed out our programs on yellow paper tape which was sent to the other high school to run. In college our physical chemistry professor decided to teach us some useful skills, so in addition to learning to solder together a crystal radio, we ran programs on punch cards also in Fortran. In grad school it was still punch cards (one audited course on Pascal). The rest is self taught.

I think time actually goes in circles so that everything is new every once

netnews access

From Rick Moen

Answered By: Raj Shekhar

I wrote:

> I was thinking TAG might be able to include or excerpt from this Usenet
> thread?

[Jimmy] Look here for the thread in question.

This is how big a newsgroup/newreader fan I am: If you check headers, you'll notice that I'm posting this with the "tin" newsreader to newsgroup "lg.tag".

[Jimmy] Quoting Rick's headers:

Newsgroups: lg.tag
Organization: If you lived here, you'd be $HOME already.
User-Agent: tin/1.7.6-20040906 ("Baleshare") (UNIX) (Linux/2.4.27-2-686 (i686))
Date: Tue, 24 May 2005 19:50:14 -0700
XRef: linuxmafia.com lg.tag:8

What's up with that, you might wonder? Well, Mailman has a feature, unused by most listadmins, to bidirectionally (or otherwise) gate any mailing list with an NNTP newsgroup. Once set up, any mailing list post shows up (across the gateway) within a few minutes, as an article in the relevant group's news spool -- and, conversely, any article posted to the newsgroup's spool likewise gets hustled the other direction across the gateway within a few minutes, and mailed out by Mailman to mailing list subscribers.

Please note distinction: I did not say the mailing list goes out on Usenet. Usenet is a (very) large system of NNTP newsgroups, but many people also operate private newsgroups that have nothing to do with Usenet (other than also relying on NetNews Transport Protocol).

In my case, I use leafnode's 2.x betas to support local newsgroups, which I don't offer to other news servers. (At this date, you have to compile 2.0 betas, as local groups are a prerelease feature not included in the 1.x release series.)

For example, my local LUG, CABAL, has a mailing list called "conspire" -- conspire@linuxmafia.com. At the time I set that up, I also compiled leafnode 2.0b7, configured it to run under inetd, and added this line to /etc/leafnode/local.groups:

  cabal.conspire	y	Local newsgroup for the CABAL Linux user group.

Then, I enabled Mailman's gateway feature for conspire@linuxmafia.com, et voila.

Basically, all I needed to do, today, was enable the gateway function for tag@lists.linuxgazette.net, and add this second line to /etc/leafnode/local.groups:

  lg.tag	y	Linux Gazette's The Answer Gang

For a very long time, I had open posting to anyone who wished to connect to my machine using NNTP. That came to an end a few months ago on account of... guess what? NNTP spammers. So, at that point I locked down access using /etc/hosts.deny (using Wiese Venema's TCP Wrappers library): Anyone wanting remote NNTP access can still get it, but you have to send me your fixed IP address (if you have one) -- and I then add a permission line to /etc/hosts.allow.

What's so great about newsgroups, you ask?

Let's say you become interested in CABAL's mailing list / newsgroup today, May 24, 2005. You can, of course, join the mailing list -- allowing you to read and respond to new posts. (Older posts are browseable only via the Pipermail archive at http://linuxmafia.com/pipermail/conspire , and you can't easily respond to them.)

Or you can participate in it, in its newsgroup form. You have instant access to all posts ever made , all the way back to December 2000 -- and can continue the thread on any of them. You don't need to subscribe before you can post, or unsubscribe to stop being barraged with stuff.

And newsgroups settled the infamous private-versus-group reply issue properly , long ago. There are no dumbass flamewars over Reply-To, because "Followups" (group responses) are treated distinctly from mailed (private) responses.

The only drawbacks are (1) people being unfamiliar with newsreaders and how newsgroups work, (2) people confusing the general concept of "newsgroups" with whatever they've heard about Usenet, and (3) people stuck behind poorly designed firewalls that don't allow NNTP access (119/tcp) but do permit e-mail.

Fortunately, adding NNTP access to a Mailman list doesn't impair "regular" mailing list functions, but does allow a second avenue of access.

One little problem I haven't figured out how to fix: Mailman sends over only to the news server only new mailing list posts, starting with establishment of the gateway. So, lg.tag's news spool has only the five or six latest TAG posts in it, whereas cabal.conspire's has that entire mailing list's history.

Ben, if you feel like a challenge (since you have a shell account, here), Mailman keeps the mailing list's mbox at /var/lib/mailman/archives/private/tag.mbox/tag.mbox, which you'll be able to read/copy, even though you can't "ls" the directory it's in. (List members can fetch that file as http://lists.linuxgazette.net/mailman/private/tag.mbox/tag.mbox .) Leafnode has the corresponding mail spool inside /var/spool/news/lg/tag/ .

If you or anyone else can figure out how to get the rest of the mbox's contents into that news spool, I'd be grateful.

[Raj] There is a service run by news.gmane.org which turns publicly available mailing lists to news (or usenet posts, I am not sure of the correct term). They have a free nntp server running at news.gmane.org and you can follow quite a lot of mailing lists using you favorite nntp client.

I find it quite useful in following mailing lists in which have a bit of interest.

Excellent tip! I note that they have 6929 newsgroups, including "gmane.org.user-groups.linux.cabal". But, of course, anything you post there isn't going to get gated back into the mailing list in question: It's a unidirectional gateway, only.

[Raj] It is bi-directional

October 2005 (#119):

The Mailbag

reminder software

Security implications of root login over SSH

/LDP/LGNET/114/misc/tag/mike.show-mtime.pl.txt

Outlook replacements

Jabberd Installation Guide Comments

Localhost considered harmful

LG 105: RSS and Feed Readers Addendum

Heather is in England...

Heather and Jim taking flight.

SVN

More 2 Cent Tips!

closing unneeded ports

TFTP problem

HTTPS question

Mozilla hogging the screen

Couple of perl related questions

Securing rsync

Boot from cd via floppy

Security implications of root login over SSH

The Answer Gang

Contents:

Converting bitmap to binary

freeshell.org service outage

netnews access