Back to Home / #linode / 2006 / 05 / Prev Day | Next Day
#linode IRC Logs for 2006-05-23

---Logopened Tue May 23 00:00:42 2006
00:12|-|Newsome [] has quit [Quit: Linux: Now with employee pricing!]
00:16|-|cvera [] has quit [Quit: CGI:IRC (Session timeout)]
01:17|-|jungleg [] has joined #linode
01:17<@caker>jungleg: hey
01:17<@caker>jungleg: your node is over 200M into swap
01:17<jungleg>hey caker!
01:17<jungleg>haha, was gonna ask
01:18<jungleg>is it the log files you think?
01:18[~]warewolf fg
01:18<@caker>jungleg: what are you running on that box? apache/mysql?
01:18<@caker>apache1 or apache2?
01:18<jungleg>apache2, I believe
01:18<@caker>ok, know what to do to reduce memory usage?
01:19<warewolf>killall -9 httpd
01:19<warewolf>er I mean ...
01:19<jungleg>is it RAM or Disk memomery I' running out of?
01:19<@caker>jungleg: RAM
01:19<jungleg>I've been getting a lot of traffic lately
01:19<@caker>memory equals RAM in my book, never disk space
01:20<jungleg>it's around 250k pageviews per month
01:20<Battousai>damn straight
01:20<@caker>jungleg: that's not the problem - it's your httpd config
01:20<@caker>jungleg: and probably mysql, too
01:20<jungleg>so I'm lost, can you point to what I can do?
01:20<@caker>jungleg: fairly easy to fix
01:20<erikh>slim down your min and max processes
01:20<erikh>avoid things like mod_perl :)
01:21<Battousai>cache it if possible ;)
01:21<@caker>yeah, run this for me: httpd -V | grep MPM
01:21<warewolf>mod_perl is perfectly fine, when configured correctly :)
01:21<erikh>eh, i worked on mod_perl for 6 years professionally
01:21<jungleg>I can't log in, can i do that from lish?
01:21<erikh>it's a beast when you're doing anything complex
01:21<@caker>jungleg: reboot first then we'll fix it, I guess
01:22<warewolf>erikh: where'd you work?
01:22<Battousai>204 days
01:22<erikh> and
01:22<Battousai>i kinda dont wanna migrate to xen any time soon
01:22<Battousai>i wanna see how long this beast lasts
01:22<warewolf>erikh- Mason based?
01:22<erikh>no, custom jobs
01:22<erikh>but I did work for a mason place for a while
01:22<warewolf>oh god
01:23<warewolf>I started doing mod_perl with all custom crap
01:23<erikh>most of the mason work I've seen is ... scary
01:23<fo0bar>caker: I just looked at opryland's web site... holy crap it is big
01:23<warewolf>thought it was slicker than shit
01:23<Battousai>whats that site with the list of all those linodes
01:23<@caker>fo0bar: it's FREAKING HUGE
01:23<warewolf>then I realised how much a pain in the ass it is to maintain
01:23<Battousai>i was gonna join it but i was at school and the cablemodems were fritzing and i forgot where it was
01:23<@caker>fo0bar: like, a neighborhood could fit inside
01:24<@caker>fo0bar: also, there's a river through it
01:24<Battousai>thats the one
01:24<fo0bar>caker: the largest hotel/casino in reno, silver legacy, has 1700 rooms. gaylord advertises nearly 3,000 rooms
01:24<warewolf>of side note: is written entirely in Mason
01:24<fo0bar>caker: I'm reading the history of it on wikipedia now
01:24<Battousai>looks kinda like html to me
01:24<jungleg>it's rebooting
01:25<warewolf>69 registered linodes
01:25<@caker>fo0bar: well, the Opry "show" used to be at the Ryman auditorium, downtown Nashville -- probably the best sounding venue I've ever been to .. they moved it over to Opryland (big hotel and mall) a few years ago
01:25<erikh>warewolf: I guess everyone is different. I just know the MF guys have soem choice words about mason after acquiring a competitor, and none of it is nice.
01:25<warewolf>with 12 that registered, and never logged any stats
01:25<@caker>(moved the show, not the auditorium)
01:26<erikh>guitar center's website.
01:26<gpd>is it just me or is 'gaylord' not the first choice of name
01:26<@caker>erikh: you play? (didn't read scrollback, sorry)
01:26<warewolf>erikh: are you on the east coast?
01:26<erikh>no, west
01:26<@caker>gpd: heh, that guy owns everything in Nashville
01:26<warewolf>I ask because I hear guitar center commercials on the radio ALL THE TIME
01:26<erikh>caker: drums, a little... I played msot of school
01:26<gpd>and... is he a 'gaylord' ? heh
01:27<erikh>oh god why did I know that was goign to come up. :)
01:27<warewolf>erikh: I've been told by friends who work for Amazon that amazon's website is backed by Mason
01:27<fo0bar>gpd: oh, come on, you can find plenty of examples like that
01:27<erikh>err, I didn't play drums in school
01:27<erikh>I played sax. :)
01:27<erikh>just learning the drums.
01:27[~]gpd hangs head in shame
01:27[~]warewolf has played trombone, baratone, and tuba
01:27<erikh>I thought Amazon was done in C++?
01:27<fo0bar>gpd: growing up, the owner of the school bus service all the school districts used was LAMERS BUS LINES
01:27<erikh>I know they do some stuff in perl, though.
01:27<warewolf>the website is written in mason
01:27<warewolf>backend stuff in c++/java
01:28<gpd>it may be a british slang thing [fo0bar - nice :)]
01:28<erikh>yeah, we had a template toolkit like thing that the content team worked with
01:28<jungleg>caker: ready :)
01:29<@caker>jungleg: httpd -V | grep MPM, paste results here
01:29<erikh>we worked between that and oracle more or less
01:31<erikh>no matter, I'm just a college bum now
01:31<@caker>... any day now ... :)
01:31<@mikegrb>caker: the gaylords own a lot of stuff in OKC too, even have a street named after them
01:32<jungleg> -D APACHE_MPM_DIR="server/mpm/prefork"
01:32<@caker>jungleg: ok
01:32<@mikegrb>fo0bar: opryland is a popular tourist attraction even for people not staying there, I've been
01:32<@caker>jungleg: edit your httpd.conf file, scroll down to the prefork section
01:32<@caker>anyone have a prefork section to paste here?
01:33<@caker>nm, got it
01:33<@mikegrb>StartServers 3
01:33<@mikegrb>MinSpareServers 2
01:33<@mikegrb>MaxSpareServers 4
01:33<@mikegrb>MaxClients 20
01:33<@mikegrb>MaxRequestsPerChild 50
01:33<@caker>jungleg: use these settings:
01:33<@caker>StartServers 5
01:33<@caker>MinSpareServers 5
01:33<@caker>MaxSpareServers 10
01:33<@caker>MaxClients 10
01:33<@caker>MaxRequestsPerChild 350
01:33<@caker>that'll fix apache
01:34<jungleg>hold on
01:34<@caker>jungleg: save your config with those settings under prefork, then restart apache
01:34<@mikegrb>10 spare servers seems quite excessive, expecially for a linode
01:34<@caker>it should say "<IfModule prefork.c>
01:35<@mikegrb>especially if the max is only 10 anyway
01:35[~]warewolf ponders purchasing a second pixelante t-shirt
01:35<warewolf>I'm wearing the original orange-on-brown shirt now
01:35<warewolf>this one with the nes controller looks great:
01:41<Battousai>my linode is rated
01:42<jungleg>caker: done
01:42<warewolf>time is money my friend
01:42[~]gpd retains 32nd spot on rml :)
01:42<jungleg>caker: should I do something similar to MySQL?
01:42<gpd>thanks to 2.6.16 weirdness
01:43<gpd>caker: anything in worth compiling a kernel for?
01:44<@mikegrb>gpd: caker confirmed it was a uml problem and came up with a work around, dunno if he has pushed it out to all the hosts yet though
01:44<gpd>mikegrb: interesting - will keep an eye out
01:46<warewolf>gpd- 2.6 weirdness?
01:46<warewolf>gpd- do explain
01:46<gpd>the load on 2.6.16 (NPTL enabled) kernel is higher than it really is
01:46<jungleg>caker: the site looks fine now -- thanxs a bunch!
01:46<gpd>my load is 0.2 even when totally idle
01:47<warewolf>I don't think load has much to do with io rate
01:47<warewolf>well, directly speaking.
01:47<gpd>what is rml reading [directly speaking] ?
01:48<warewolf> /proc/io_status
01:48<warewolf>to date, I have received $0.00 in donations for RML :/
01:48<warewolf>I really should finish up rml 2.0 and chuck google ads on it
01:48<gpd>io_count=451839427 io_rate=0 io_tokens=400000 token_refill=512 token_max=400000
01:49<gpd>mostly 0...
01:49<gpd>and yet 32nd place
01:49<warewolf>don't watch io_Rate
01:49<warewolf>watch io_count
01:49<warewolf>it's a counter
01:49<warewolf>that's what rml monitors
01:49<gpd>how can your count go up with 0 rate?
01:49<warewolf>io_rate is actually quite useless
01:49<gpd>it seems to be doing so
01:49<warewolf>beacuse that io_rate is a snapshot of how much io you're doing that very split second
01:50<warewolf>which, unless you're thrashing, has a high liklihood of being near-zero.
01:50<warewolf>io_count=154016454 io_rate=49 io_tokens=399936 token_refill=512 token_max=400000
01:50<warewolf>that's mine
01:50<warewolf>io_count=154016531 io_rate=34 io_tokens=400000 token_refill=512 token_max=400000
01:50<warewolf>and again
01:50<warewolf>see how io rate is 34 but I have full tokens?
01:51<gpd>top - 23:51:02 up 48 days, 15:16, 6 users, load average: 0.30, 0.31, 0.26
01:51<gpd>Cpu(s): 0.0% us, 0.0% sy, 0.0% ni, 100.0% id, 0.0% wa, 0.0% hi, 0.0% si
01:51<warewolf> 02:51:08 up 57 days, 23:49, 3 users, load average: 0.40, 0.54, 0.57
01:51<gpd>100% idle, but 0.3 load...
01:51<warewolf>my average io/sec is 30 for this boot
01:52<gpd>Sorry that this isn't very definite of an answer, but I think with NPTL threads are displayed differently in things like ps, so that could account for loadavgs appearing larger. <-- caker
01:53<warewolf>loadaverage really sucks for benchmarking stuff
01:53<Battousai>yeah, my load avgs are around 0.4 or so perpetually
01:53<warewolf>you really want to monitor cpu time
01:54<warewolf>I don't even monitor load average
01:54<gpd>well something has to explain 32nd place in rml
01:54<gpd>which suddenly rose after changing to that kernle
01:54<gpd>and also reported by others in that thread
01:55<gpd>and mikegrb ^^ says caker agrees...
01:55<warewolf>gpd- what's your linode name? feel free to /msg it to me if it isn't public
01:55<gpd>34th place = gravies
01:55<gpd>obviously 2 new ppl
01:55<@mikegrb>gpd: no, I was referring to linodes shuting down 20 seconds after boot most of the time
01:56<gpd>well i am clearly using mine more than most then!
01:56<gpd>i suppose it is doing imap every 5 mins + apache + bind + postfix + 2x irssi + stuff
01:57<Battousai>all mine does is one very boring lighttpd
01:57<gpd>+ monit + munin... + mysql + supybot...
01:57<Battousai>and an irssi
01:57<@mikegrb>gpd: what is your io_count / seconds of uptime
01:57<@mikegrb>an average value is about 10
01:58<gpd>io_count=451882202 23:58:04 up 48 days, 15:23
01:58<@mikegrb>cat /proc/uptime
01:58[~]warewolf is 9th worst ranked on rml
01:58<gpd>4202648.21 3501218.67
01:59<@mikegrb>!calc 451882202 / 4202648.21
01:59<linbot>mikegrb: 451,882,202 / 4,202,648.21 = 107.523204
01:59<gpd>so i am a monster :|
01:59<gpd>but not a thrasher :)
02:00<@mikegrb>most thrashers are about 50-100
02:00<gpd>easy now
02:00<@mikegrb>so you may not be hitting swap heavily, but you are doing the same kind of io
02:00<gpd>i am not hitting swap at all
02:00<gpd>i have 10M free most of the time
02:00<warewolf>I think it was gpd who said he was getting heavy web traffic
02:01<warewolf>oh no that was jungleg
02:01<gpd>i suspect munin
02:01<erikh>that reminds me
02:01<@mikegrb>but junleg wasn't getting heavy web traffic
02:01<@mikegrb>only about 5 hits / minute
02:01<gpd>well Linux FAILS to have any tools to find out what is causing I/O by process :(
02:02<warewolf>gpd- there's two issues behind that
02:02<warewolf>1) swap
02:02<erikh>nothing like top -m
02:02<warewolf>2) regular io
02:02<erikh>on freebsd
02:02<gpd>erikh: we've been through this - not that i can find
02:02<warewolf>a process can chew memory to the point where it swaps -- that's the kernel doing IO then
02:02<gpd>apparently - 'it is difficult'
02:02[~]erikh needs to configure snmp
02:02<erikh>gpd: yeah, I imagine it's no easy task
02:02<warewolf>otherwise it's the process, which could be accounted with utilities
02:03<gpd>supybot seems to be using 58M or virtual :(
02:03<erikh>or were you talking about snmp?
02:04<erikh>snmp isn't hard.
02:05<erikh>configuring traps can be
02:06<gpd>apart from `watch cat /proc/io_status`... what can i use to monitor?
02:07<gpd>io -- bi and bo = 9 ? wtf that means
02:08<warewolf>blocks in
02:08<warewolf>blocks out
02:08<warewolf>although there are more specific values you probally want to watch
02:08<gpd>seems to be static at 9
02:08<gpd>giong back to io_status
02:08<warewolf>mmmm.. I Forget where they are
02:09<gpd>ok -stopping everything...
02:10<gpd>ok just irssi, screen, sshd running now
02:11<warewolf>is irssi logging to ~/IrcLogs ?
02:11<warewolf>'cause that will generate io aswell
02:11<gpd>not sure - how do i find out?
02:12|-|gpd_ [] has joined #linode
02:12<warewolf>is there an irclogs directory in your homedir?
02:12|-|gpd [~gpd@] has quit [Quit: leaving]
02:12<warewolf>then it doesn't appear to be logging
02:12<gpd_>ok that is gone too :)
02:13<gpd_>still going up quite rapidly
02:14<gpd_>io_count=451966478 io_rate=0 io_tokens=400000 token_refill=512 token_max=400000
02:14<gpd_>Every 10.0s: cat /proc/io_status Tue May 23 00:14:15 2006
02:14<gpd_>io_count=451966507 io_rate=0 io_tokens=400000 token_refill=512 token_max=400000
02:14<@caker>gpd_: syslog, klog, etc?
02:15<gpd_>stop them?
02:15<gpd_>or possible cause?
02:16<warewolf>possible cause
02:16<jungleg>good nite, all
02:16<gpd_>caker: are you going to build 2.6.17-4 or ?
02:16<gpd_>maybe i should try 2.6.17-rc2
02:16<@caker>gpd_: just going to wait for 2.6.17 final
02:16<@caker>gpd_: why?
02:16<@caker>what are you trying to figure out?
02:16<gpd_>well i have a feeling all this is due to NPTL
02:17<gpd_>why i am 34th on rml
02:17<@mikegrb>gpd_: those two values are only 29 apart
02:17<@caker>all what, exactly?
02:17<@mikegrb>over 10 seconds
02:17|-|jungleg [] has quit [Quit: jungleg]
02:17<@caker>yeah, that's nothing
02:17<@mikegrb>which is only an average of 2.9/sec
02:17<@mikegrb>which is quite alright
02:17<@mikegrb>much better then the 100+ it was
02:18<@caker>mikegrb: also, the umid_tree_recreate() fix is only on host56 and up
02:18<@caker>mikegrb: tomorrow I'm going to bring the rest of the hosts to the unified xen+uml host-stuff
02:19<@mikegrb>caker: yeah, I suspected that
02:20|-|gpd [~gpd@] has joined #linode
02:20|-|spr [] has quit [Ping timeout: 480 seconds]
02:21|-|gpd_ [] has left #linode []
02:25|-|spr [] has joined #linode
02:30|-|Ciaran [] has quit [Ping timeout: 480 seconds]
02:31|-|spr [] has quit [Quit: Spoon!]
02:36<Battousai>May 23 03:36:27 fenrir mysqld[19508]: mysqld got signal 11;
02:37[~]Battousai kicks ubuntu
03:02|-|agundur [] has joined #linode
03:14|-|Dreamrr_3 [] has quit [Read error: Connection reset by peer]
03:19|-|Ciaran [] has joined #linode
03:49|-|konoko [] has quit [Ping timeout: 480 seconds]
04:24|-|Spads [~crack@] has joined #linode
04:45|-|agundur [] has left #linode [So Long, and Thanks for All the Fish!]
05:11|-|womble [] has joined #linode
05:34|-|jekil [] has joined #linode
06:36|-|womble [] has quit [Quit: What's behind the round window...]
07:14|-|Dreamer3 [] has joined #linode
07:18|-|linville [] has joined #linode
07:35|-|Quizzer [] has quit [Quit: anonymous user has left irc...]
07:41|-|jekil [] has quit [Quit: Leaving]
07:54|-|Quizzer [] has joined #linode
08:01|-|htranou [] has joined #linode
08:02<htranou>needed some help about the activation of my account
08:14<encode>htranou: its probably the wrong time of day - the last activity in here from someone that could do that for you was around 6 hours ago
08:15<htranou>hmmm, guess I'll have to wait then
08:46<htranou>btw, how can I access the cpuinfo?
08:46<htranou>/proc/cpuinfo doesn't show the features
08:46<encode>what cpuinfo did you want?
08:47<htranou>I'd like to know the cpu features. Like mmx sse...
08:53<anderiv>htranou: I'm guessing you got your account activated, then...
08:54<htranou>yes, but I don't know if I still have to send a copy of my CC
08:54<htranou>haven't sent it yet, but they still have activated my account
08:55<htranou>would be annoying if they disable the account tomorrow
08:56|-|FireSlash [] has joined #linode
09:32<npmr>htranou, no, if you're in, you're in
09:32<npmr>there are a lot of verifications (copy of CC, phone call, etc) that they will only do as a last resort
09:33<npmr>most people never have to do those
09:33<htranou>they did send me a mail saying I had to send those
09:33<htranou>so, I found it strange, when I got the mail confirming the activation
09:33<npmr>so, /proc/cpuinfo isn't telling you about the cpu because as far as your linoe's kernel knows, the host system's kernel is the cpu
09:33<npmr>well anyway, your account is active now
09:35<htranou>I still would like to know the exact specs of the cpu, since I will compile things from my own computer
09:35<htranou>don't want to have binaries that can't run there
09:35<npmr>use that and you'll be fine
09:36<htranou>I thought to put at least i686
09:36<npmr>i'm pretty sure they all are, yes
10:07|-|FireSlash [] has quit [Read error: Connection reset by peer]
10:25<htranou>it's running an opteron it seems
10:45|-|Battousai [] has quit [Quit: Leaving]
10:46|-|Battousai [] has joined #linode
10:56<linbot>New news from forums: Howto for Simple Mail Server on Debian Would be Appreciated in Email/SMTP Related Forum <>
10:57|-|botah [~botah@] has joined #linode
10:58<botah>hi... im trying to get a LAMP server setup
10:58<botah>whats the best distro for that?
10:58<anderiv>botah: whatever distro you're most comfortable with.
10:58<botah>well RH i suppose
10:59<anderiv>botah: give Fedora Core a try,'s free and based off of Redhat.
10:59<botah>i guess i was speifically asking if there is a distro that is 'newer' ie: latest mysql php and the such
10:59<gpd>botah: ubuntu will be the newest
11:00<botah>ive been speding all day upgrading rh
11:00<gpd>it is debian based and easy to upgrade
11:00<gpd>dapper drake comes out next week and should be solid
11:00<gpd>redhat->debian-based is fairly easy transition
11:00<anderiv>yup...ubuntu's a good one...
11:00<botah>ubantu... do you use rpms to upgrade
11:01<gpd>no debian apt-get
11:01<anderiv>botah: deb packages.
11:01<gpd>apt-get update && apt-get dist-upgrade - job done
11:01<gpd>about 40-50% of linodes are debian
11:01<gpd>centos is also an option if you want DeadRat
11:01<botah>so i can upgrade my whole distro in one shot
11:02<gpd>yes - pretty much
11:02<anderiv>botah: one of the *major* problems with most rpm-based distros is horrible package dependency management. Debian (and hence ubuntu's) apt system solves that problem.
11:02<botah>tell me about it
11:02<botah>i could kill myself now
11:02<gpd>i quit redhat at 9 and never looked back
11:02<botah>i spent all day yesterday dicking with it
11:02<anderiv>gpd: hehe ditto.
11:02<anderiv>botah: yah - give ubuntu or debian a try.
11:03<botah>ubuntu it is
11:03<anderiv>gpd: did caker give you that figure about the % of linodes on debian?
11:03<botah>one cool thing about linode is how easy it is to 'resstall' distros
11:04<gpd>anderiv: mikegrb had a pie chart a while ago
11:04<anderiv>interesting...never saw that.
11:04<anderiv>I've been contemplating a gentoo->debian move myself. I'd have to get a temporary "intermediary" linode for the transition, though. Can't afford the downtime.
11:10<botah>so... i got ubantu now... just `apt-get install apache`???
11:10<anderiv>botah: apt-get update first....
11:10<botah>or should i start w/ `apt-get dist-upgrade`
11:10<botah>ahh yes
11:10<botah>you beat me
11:10<anderiv>botah: that'll make sure you have the latest package defs.
11:10<anderiv>err...whatever you call 'em.
11:11<botah>opps we had diff commands
11:11<botah>should i also run dist-upgrade?
11:11<botah>or just update
11:11<anderiv>apt-get update first.
11:12<anderiv>that will just download the latest list of what packages are available.
11:12<anderiv> may also want to add some lines to your apt sources file depending on what packages you'll want to install.
11:13<anderiv>...that's about the extent of my apt knowledge, though.
11:13<anderiv>use apt-cache search <package_name> for searching available packages.
11:14<botah>i dont see apahce 2.0.XX in there
11:14<botah>i dont want apache2
11:16<TheFirst>ummm apache2 is 2.0.xx ...
11:17<botah>i thought 2.2.x was
11:17<TheFirst>but unless you need to i'd suggest running a lower resource httpd than apache
11:19[~]Battousai consumer whore
11:21|-|botah [~botah@] has quit [Read error: Connection reset by peer]
11:24|-|botah [~botah@] has joined #linode
11:27<anderiv>botah: welcome back
11:27<anderiv>botah: I second the vote for lighttpd...I'm currently migrating to that from apache
11:29<botah>how about support for php
11:29<anderiv>botah: check
11:30<anderiv>botah: I could say with quite a bit of certainty that lighttpd will support just about anything you need it to do.
11:31<botah>how about tomcat or resin java servers?
11:31<Quizzer>never was able to lighttpd working properly on ubuntu
11:31<botah>ie: are there plugins for those?
11:32|-|jekil [] has joined #linode
11:32<TheFirst>lihttpd does php through fcgi
11:32<TheFirst>runs horde and wordpress fine...though the later did need a rewrite to make it work perfect
11:38<anderiv>Quizzer: what weren't you able to get working?
11:38<anderiv>botah: I'm not sure about the java servers....I don't touch that stuff :-)
11:38<anderiv>botah: hit up the lighttpd website.
11:38<Quizzer>never started up as a server
11:39<Quizzer>tried following one of the tutorials i found via google
11:39<Quizzer>about how to get lighttpd and ubuntu working together
11:40<TheFirst>from the lighty forums it looks like you can use tomcat through mod_proxy
11:52|-|Barqs [~jesse@] has joined #linode
11:53|-|Barqs [~jesse@] has left #linode []
12:08<botah>hmmm..apt-cache search lighttpd
12:08<botah>returns nothing... do i have to install by hand then?
12:27<@caker>!weather 37216
12:27<linbot>caker: Temperature: 73°F / 23°C | Humidity: 38% | Pressure: 30.17in / 1022hPa | Conditions: Partly Cloudy | Wind Direction: Variable | Wind Speed: 4mph / 6km/h; Today - Mostly sunny. Highs around 80. Northeast winds around 5 mph.; Tonight - Mostly clear. Lows in the lower to mid 50s. Northeast winds 5 mph in the evening...then becoming light and variable.; Wednesday - Partly cloudy. Highs in the mid 80s. Southwest (1 more message)
12:27[~]caker drools
12:29|-|botah [~botah@] has quit [Ping timeout: 480 seconds]
12:32|-|spr [] has joined #linode
13:14<gpd>looks like lighttpd is in dapper or debian unstable
13:29|-|linville [] has quit [Remote host closed the connection]
13:30|-|linville [] has joined #linode
13:50<TheFirst>or testing
14:09[~]npmr wonders how much one really gains by using lighttpd instead of just learning to config apache well
14:11<gpd>npmr: agreed, but I've not tried lighttpd as yet
14:11<npmr>May 23 14:57:01 orion sshd[30043]: Failed password for illegal user guanine from ::ffff: port 42698 ssh2
14:11<npmr>May 23 14:57:05 orion sshd[30050]: Failed password for illegal user adenine from ::ffff: port 42893 ssh2
14:12<npmr>May 23 14:57:07 orion sshd[30053]: Failed password for illegal user cytosine from ::ffff: port 43043 ssh2
14:12<npmr>^-- interesting
14:12<gpd>selfish DNA - starting to take over the world :)
14:12<gpd>is there an easy way to see how much memory a process eg. apache2 is using (other than totalling from top)?
14:13<npmr>and apache2 isn't so simple
14:13<npmr>if you're using mpm-prefork, you calculate it the same as apache 1.3
14:14<npmr>but mpm-worker is threaded, so all of those apache server processes are actually sharing a single vma
14:14<npmr>even though in top they look like separate processes
14:15<npmr>see also: slapd, named (bind version 9 and up), asterisk
14:27|-|botah [] has joined #linode
14:42|-|botah [] has quit [Quit: botah]
15:08|-|jekil [] has quit [Ping timeout: 480 seconds]
15:09|-|coumbes [~coumbes@] has joined #linode
15:24<erikh>is there something wrong with host60.fremont?
15:24<erikh>my second question: do caker or mikegrb ever sleep?
15:24<anderiv>they're bots :-)
15:25<iggy>mikegrb slept till almost 2 in the afternoon today
15:25<iggy>lazy bum
15:27<@caker>erikh: host60 looks fine to me, why?
15:27<erikh>it might be my connection. I get intermittent pings and can't shell in on occasion
15:27<erikh>my io_status looks alright
15:31<erikh>it seems like it's a problem with establishing connections, not keeping them.
15:31<erikh>and I have no rules loaded
15:31<heidi>iggy: but he was up until 5 am
15:33<iggy>heidi: he should stop that
15:41|-|linville [] has quit [Quit: Leaving]
15:46<gpd>apache's memory load seems to be heavily dependent on the number of virtual servers
15:46<gpd>I just removed a few that i am not using and it went from 30M VIRT to 15M
15:46<gpd>Not sure if there is a setting that can influence that
15:47<gpd>253 root 16 0 15104 5180 3188 S 0.0 4.5 0:00.02 apache2
15:47<gpd>254 www-data 18 0 15104 2884 880 S 0.0 2.5 0:00.00 apache2
15:47<gpd>+ 2 more lke the 2nd line
15:48<gpd>making a total of 8064 if you assume the 3 www-data ones are sharing the 2884
15:49<gpd>npmr: is that correct for apache2-mpm-prefork?
15:54|-|spr [] has quit [Quit: "Ah, the bounce has gone from his bungie"]
16:04<npmr>the memory usage number that matters in this case is the size of virtual memory
16:04<npmr>that's the VIRT column in top, or VSZ in 'ps aux'
16:05<npmr>it will almost always be partially paged out, so that's not really too concerning
16:06<@caker>TheFirst: around?
16:06<npmr>but basically, virtual memory size minus shared memory should get you roughly how much the process is using in heap
16:06<npmr>which is the part that you can usually reduce
16:07<TheFirst>caker: sure, why?
16:07<@caker>TheFirst: I jsut saw your node thrashing a bit
16:07<@caker>TheFirst: I think it's about 115M into swap or so
16:07<npmr>with mpm-prefork you take all of the heaps of all of your apache processes, plus the size of your apache2 binary, plus the sizes of your apache modules, and that's how much memory apache uses
16:07<TheFirst>not mine
16:07<TheFirst>Mem: 80 76 3 0 7 31
16:07<TheFirst>-/+ buffers/cache: 38 41
16:07<TheFirst>Swap: 256 0 256
16:07<TheFirst>scare the hell out of me :P
16:08<npmr>you may also want to factor in the memory usage of any cgi scripts that your server is running
16:08<TheFirst>thought i had php or something run amock
16:08<npmr>but those aren't technically a part of any apache process
16:09<@caker>TheFirst: you don't have a Linode with the same username as your nick?
16:09<@caker>ok then
16:09<@caker>yeah, figured there was another
16:09<TheFirst>some bastard that hasn't been on here in years regged gaveup and it seems they dont expire nicks :(
16:10<@caker>OK, new host-libraries deployed to all the hosts... about a year's worth of work (mostly into Xen)
16:10<TheFirst>so you've updated xen again?
16:10<TheFirst>or are you deploying xen all out?
16:10[~]TheFirst hopes not
16:10<@caker>no, it's the unified uml+xen host libs that went out, along with bug fixes for all
16:11<TheFirst>speaking of thrashing...someone's doing it on 56
16:11<TheFirst> Timing buffered disk reads: 8 MB in 3.05 seconds = 2.62 MB/sec
16:18<gpd>ok this is weird - i did mv /lib/tls /lib/tls.disabled...
16:19<gpd>and restarted stuff - and now lsof shows it is using it in /lib/tls.disabled!
16:20|-|thoth39 [] has joined #linode
16:20<npmr>you know that last 2.6 kernel supports tls
16:20<@caker>and it boots correctly now, too
16:20<gpd>yeah - that is what i am running...
16:21<gpd>but i have this weirdly high io
16:21<gpd>and wondering if that is the cause
16:21<gpd>seems to keep spiking to >1000
16:21<gpd>giving me an average of about 200-300
16:21<gpd>are there tools to monitor io_status to get a real io rate?
16:22<gpd>the rate given doesn't seem to reflect the actual increase in total io
16:24<@caker>gpd: io_count is taken right from uml's io thread, and io_rate is calculated each second. It's accurate
16:25|-|Spads [~crack@] has quit [Ping timeout: 480 seconds]
16:25<lucca>gpd: the libs are still in use, where-ever you move them to. Likely the offending process is init, so you'll need to tell it to re-exec or else reboot
16:26|-|konoko [] has joined #linode
16:27<gpd>lucca: so will telinit u do the trick?
16:30<linbot>New news from forums: Kernel: 2.6.17-rc2-linode19 (with NPTL/TLS support) in Announcements <>
16:32<lucca>gpd: yeah
16:32<gpd>didn't seem to - guess i'll have to reboot
16:40|-|timber [] has joined #linode
16:44|-|Dreamer3 [] has quit [Remote host closed the connection]
16:45|-|Dreamer3 [] has joined #linode
16:46|-|timber [] has quit [Quit: CGI:IRC (Ping timeout)]
17:00|-|jekil [] has joined #linode
17:13|-|Damaja1 [] has joined #linode
17:35|-|Spads [~crack@] has joined #linode
17:37<gpd>looks like kjournald keeps appearing with D state... maybe that is the spiker
17:41|-|timber [] has joined #linode
17:42|-|womble [] has joined #linode
17:57|-|jekil [] has quit [Ping timeout: 480 seconds]
18:05<gpd>and munin seems to be writing a lot of log crap
18:06<@caker>it enjoys doing that
18:06<gpd>I set log_level 0 in munin-node.conf - no difference
18:07<@caker>you've got the collector on the same box?
18:07<@caker>there should be a munin.conf then
18:07<@caker>probably log settings for that, as well
18:08<@caker>also, you're 90M into swap, if you're still hunting for IO stuff
18:09<gpd>ok stopping stuff
18:13<gpd>ok i have no idea what is using all the ram|swap
18:19<gpd>caker: when you say 90M into swap... that isn't necessarily bad is it?
18:19<gpd>i am now 10M into swap and have 14M free
18:19<@caker>gpd: correct
18:20<@caker>gpd: but, having 90M in swap increases the chance that things are swapping in/out causing IO
18:20<@caker>since you're on your IO hunt
18:20<gpd>so... in summary... there isn't much I can do about it... ?
18:20<@caker>I'm sorry, about what?
18:21<gpd>about the fact that i am averaging 100 io/sec since boot...
18:21<@caker>I'm still unclear as to what problem you're trying to solve
18:21<gpd>why I am 36 out of 36 on RML
18:21<gpd>[not including you]
18:21<@caker>ok, was going to say :)
18:21<@caker>well ... lemme think for a sec
18:21<gpd>either nobody actually does anything on their linodes...
18:21<gpd>or there is something wrong with mine
18:22<@caker>gpd: what "user" are you listed as on RML?
18:23<@caker>I'd look at swap activity first...
18:23<@caker>vmstat 1 <--- your friend
18:24<@caker>that will at least tell you if it's swap or stuff just hitting disk
18:24<gpd>si & so ?
18:24<@caker>yeah, vs bi/bo
18:24<@caker>actually, it'll appear in both for swap, I believe
18:24<gpd>not much happening atm
18:24<gpd>seems to be 'spikes'
18:25<@caker>another thing you can do on 2.6: echo 10 > /proc/sys/vm/swappiness
18:25<gpd>possibly cron related - eg. munin
18:26<@caker>yeah, and generating graphs and other cron stuff might push things in/out of swap, too
18:26<gpd>do you have any cli tools to monitor io_status? eg. running average of io_rate?
18:27<@caker>no, that's left up to the user :)
18:27<@caker>mrtg/munin would do that nicely
18:27<gpd>yeah - i wanted something a little more lightweight
18:28<@caker>mrtg and munin scripts --^
18:28<gpd>like watch cat /proc/io_status | awk ??? | wtf | foo
18:28<gpd>yeah - i have those running
18:28<gpd>but as i suspect munin is the culprit...
18:28<@caker>You could collect them with that method, and then avg them later
18:29<@caker>cron every minute to awk /proc/io_status
18:29<@caker>append to a text file, etc...
18:29<gpd>yeah - just thought you might already have a thingy to do that...
18:29<gpd>clearly you are playing your cards close to your chest ;)
18:32|-|spr [] has joined #linode
18:36|-|timber [] has quit [Quit: CGI:IRC (Ping timeout)]
18:46|-|htranou [] has quit [Quit: ]
18:51|-|Damaja1 [] has quit [Ping timeout: 480 seconds]
19:12|-|cow [Ap0ll0@] has quit [Read error: Connection reset by peer]
19:12|-|gpd [~gpd@] has left #linode []
19:12|-|cow [] has joined #linode
19:13|-|gpd [~gpd@] has joined #linode
19:16|-|Dreamr3 [] has joined #linode
19:22|-|Dreamer3 [] has quit [Ping timeout: 480 seconds]
19:30<gpd>67 8 0 25 0 0|11.5M 60k| 672B 4066B| 0 0 |3698 405
19:30<gpd>that's it...
19:30<gpd>every few minutes - massive disk read...
19:34<@caker>what's that output from?
19:34<gpd>very handy :)
19:34<gpd>usr sys idl wai hiq siq|_read write|_recv _send|__in_ _out_|_int_ _csw_
19:34<gpd>i got rid of some crap in /etc/cron.d ... now it looks better
19:35<gpd>load is now 0.02! for the first time in months
19:35<gpd>also had a 30M bind log file - now gone
19:36<gpd>[can you tell i'm avoiding work!]
19:39<gpd> 0 7 0 93 0 0|6205k 44.6M| 208B 710B|1376k 47.7M|13.7k 681
19:39<gpd>hmm... 44M written... right during cron...
19:42<Battousai>down to 21st already
20:22|-|gpd [~gpd@] has left #linode []
20:25|-|gpd [] has joined #linode
20:30<gpd>caker: more weirdness - rebooted into .17-rc2
20:31<gpd>now if i do vmstat 1 - it scrolls past like a MOFO
20:31<gpd>vmstat 2 however seems fine ???
20:37[~]gpd ponders nanosecond timers
20:41|-|gpd [] has left #linode []
20:42|-|gpd [~gpd@] has joined #linode
20:45<@caker>gpd: the vmstat 1 oddness was with .17-rc2?
20:45<gpd>and with dstat
20:45<gpd>and even more odd -- dstat just stops after x time
20:45<gpd>but ^c breaks it
20:45<gpd>and drops back to shell
20:46<gpd>if you set dstat 10 the line stays static for 10 seconds (ish) but data keeps updating on a single line (eg. using \r)
20:46<gpd>quite freaky to watch
20:46<gpd>i think i'm going back to .15
20:46<gpd>unless you want me to test stuff
20:47<@caker>what kernel are you in now?
20:47<@caker>gpd: /join #uml
20:48<@caker>might take a few for jdike to return
20:48<gpd>k - no problem
20:51|-|egatenby [] has joined #linode
20:52<egatenby>caker? mikegrb?
20:56|-|taupehat [] has quit [Ping timeout: 480 seconds]
20:56|-|TheFirst [] has quit [Ping timeout: 480 seconds]
21:02|-|luis_ [~fortytwo@] has joined #linode
21:04|-|luis [~fortytwo@] has quit [Ping timeout: 480 seconds]
21:06|-|egatenby [] has left #linode []
21:07<Eman>lish is acting stupid on host48
21:08<@caker>define stupid
21:08<Eman>either it wont let me login, or when it does, says there is no screen session to resume
21:09<@caker>you might have to wait until I update that box, but busy dealing with host56 at the moment
21:21|-|FireSlash [] has joined #linode
21:32|-|FireSlash [] has quit [Quit: Leaving]
21:33|-|FireSlash [] has joined #linode
21:33|-|taupehat_ [] has joined #linode
21:34<taupehat_>ping caker mikegrb
21:38|-|kiomava [] has joined #linode
21:38<kiomava>anybody know whats up with host56
21:38<kiomava>my linode is shut down and not booting up
21:39<gpd>caker: is working on 56
21:39<kiomava>oh ok
21:42<thoth39>host56 is back on.
21:43|-|taupehat [] has joined #linode
21:44|-|taupehat_ [] has left #linode []
21:48|-|thoth39 [] has quit [Quit: Leaving]
21:50<gpd>caker: going to reboot unless you have further inquiries
21:50<@caker>gpd: go for it
21:50|-|gpd [~gpd@] has quit [Quit: bye...]
21:53|-|gpd [] has joined #linode
21:54|-|FireSlash [] has quit [Quit: Leaving]
21:54<gpd>not sure why i get 'fail' when starting OpenBSD Secure Shell server
21:58<@caker>gpd: move /lib/tls out of the way?
21:58<gpd>caker: nothing to do with that
21:58<gpd>ssh is fine - just that message on boot
22:00<gpd>ok vmstat and dstat back to normal on 2.6.15-linode16, tls.disabled
22:03|-|gpd [] has left #linode []
22:04|-|gpd [~gpd@] has joined #linode
22:06<gpd>sshd[2799]: Setting tty modes failed: Invalid argument
22:06|-|kiomava [] has quit [Quit: CGI:IRC]
22:06<gpd>looks like that is the fail message
22:06<@caker> ?
22:07<gpd>yeah - was just reading that
22:08|-|TheFirst [] has joined #linode
22:10<gpd>Experimenting with
22:10<gpd>client side stty settings I found that it can be caused by "stty
22:10<gpd>parenb" on the client side. "stty -parenb" stops the error message.
22:11<gpd>i have no idea what that means
22:13<linbot>New news from forums: Reboot: host56 (2006-05-23) in Xen Public Beta <>
22:14<@caker>gpd: what's your client side? Linux?
22:14<@caker>run that command before sshing
22:15<gpd>invalid argument
22:16<gpd>stty -parenb -- fine
22:16<gpd>[-]parenb generate parity bit in output and expect parity bit in input
22:16<gpd>oh well - forgetaboutit again i think
22:59|-|VS_ChanLog [] has left #linode [Rotating Logs]
22:59|-|VS_ChanLog [] has joined #linode
23:08|-|Battousai [] has quit [Quit: Leaving]
23:30<taupehat>oy vey
23:31<taupehat>those of you who do end-user support will find this one interesting, or perhaps taken straight out of Chronicles of George
23:31<taupehat>I have been trying to put the life touch pictures into school master. I think the problem is I dont have http 404 when I open internet. can you fix my internet opening and see what I'm missing?"
23:44|-|FireSlash [] has joined #linode
---Logclosed Wed May 24 00:00:26 2006