--- | Log | opened Thu Feb 28 00:00:23 2019 |
00:50 | -!- | ewsc [~oftc-webi@105.112.86.219] has joined #linode |
00:50 | -!- | ewsc is "OFTC WebIRC Client" on #linode |
00:50 | <ewsc> | Hello i am a new client |
00:50 | <millisa> | Greetings |
00:50 | -!- | ewsc [~oftc-webi@105.112.86.219] has quit [autokilled: This host may be infected. Mail support@oftc.net with questions. BOPM (2019-02-28 05:50:52)] |
00:51 | <millisa> | :( That prince was going to be my friend |
00:51 | <Unit193> | Alas, it was never meant to be. |
00:51 | -!- | fayis [~oftc-webi@103.65.33.255] has joined #linode |
00:51 | -!- | fayis is "OFTC WebIRC Client" on #linode |
00:51 | <fayis> | helllo |
00:52 | <millisa> | greetings |
00:52 | <fayis> | I am facing some issues with the linode droplets |
00:52 | <nate> | BOPM? Jesus I don't even remember the last time I saw a BOPM reference |
00:52 | <nate> | fayis: Define issues and in detail? :P |
00:52 | <millisa> | linodes are linodes. digital ocean are the ones with droplets. |
00:53 | <fayis> | I just cant access any service but the droplet is running |
00:53 | <fayis> | also I have tried white listing the ports |
00:53 | <fayis> | can I get your contact number please |
00:53 | <fayis> | I am from india |
00:53 | <millisa> | Have you used lish to get on the console and check things from there? |
00:53 | <nate> | That's not likely something they'll directly assist with unless you're paying for managed/professional services. Are you actually -in- the linode now? |
00:53 | <nate> | ie; SSH'd in? |
00:54 | <fayis> | I can access the console via the linode portal |
00:55 | <fayis> | inbuilt web console only |
00:56 | <millisa> | when you try to connect via ssh, what does it say? |
00:56 | <nate> | Try temporarily turning off whatever firewall of choice on your selected distro and see if that resolves anything, if it does then you know to double check your rules |
00:56 | <fayis> | I could see that there were more than 20k failed attempts |
00:56 | <fayis> | I think someone trying to hack my server |
00:56 | <millisa> | that's pretty normal for something on the internet with a listening ssh port |
00:56 | <nate> | technically automated bots are trying to brute force in, fairly normal if you kept the SSH port on 22 |
00:57 | <fayis> | I am using the dom9 firewall on top of firewalld |
00:57 | <fayis> | I have not kept it open but only for my ip |
00:59 | -!- | WhizzWr [Whizz@000276f4.user.oftc.net] has quit [Quit: Bye!] |
00:59 | <nate> | "on top of"? You mean Dome9? I'm not familiar with a "Dom9". Perhaps get Dome9 out of the way first and see if that resolves things. (that's not even a locally installed model is it? Isn't that like cloudflare if I recall?) |
00:59 | -!- | WhizzWr [Whizz@000276f4.user.oftc.net] has joined #linode |
00:59 | -!- | WhizzWr is "Nothing is real" on #redditprivacy #pcl #oftc #linode |
01:00 | <fayis> | yeah it is a kind of firewall |
01:00 | <fayis> | right now I tried to disable firewalld |
01:01 | <fayis> | I have been using dome9 since I spun up the server |
01:01 | <fayis> | and never faced such an issue |
01:02 | <fayis> | wont they help if the issue is with the server |
01:02 | <millisa> | It doesn't sound like it is |
01:02 | <millisa> | What is the IP of the linode |
01:03 | <nate> | It's most likely not "the server". Are you trying to access directly by IP or through your DNS protected by Dome9? If the latter, try the former, and share it here also. |
01:05 | <fayis> | I cant access directly via IP also |
01:05 | <fayis> | only I can do to connect is using their web console only |
01:05 | <fayis> | and I have cpanel installed in the server |
01:06 | -!- | NomadJim__ [~Jim@2001:5b0:2d1f:8328:11f3:9eb8:df8c:9a35] has quit [Quit: Leaving] |
01:07 | <fayis> | 66.228.42.5 |
01:08 | <fayis> | this is the IP |
01:09 | <millisa> | i can ping it. ssh looks open. |
01:09 | -!- | NomadJim [~Jim@2001:5b0:2d1f:8328:98a1:d656:aa40:5f7b] has joined #linode |
01:09 | -!- | NomadJim is "Nomad" on #linode |
01:09 | <fayis> | yes it happened now |
01:09 | -!- | thiras [~thiras@195.174.215.70] has quit [Ping timeout: 480 seconds] |
01:10 | <fayis> | I think I have messed up something in firewall |
01:10 | <millisa> | the reverse for it looks weird though... are you sure that is your IP and not a name server? |
01:10 | <fayis> | it is my IP |
01:10 | <fayis> | I am sure |
01:10 | <millisa> | then you are wrong. |
01:10 | <millisa> | that's one of the nameservers in newark. |
01:11 | <fayis> | sorry |
01:11 | <fayis> | let me know if you have any suggestions ? |
01:11 | <millisa> | use your ip |
01:11 | <fayis> | I am very sorry |
01:11 | <fayis> | 69.164.207.134 |
01:11 | <fayis> | this is the IP |
01:12 | <fayis> | it was my mistake |
01:12 | <fayis> | could you please check this ip 69.164.207.134 |
01:13 | <millisa> | it pings, looks like most ports are filtered |
01:13 | <fayis> | can you ping ssh port also ? |
01:14 | <fayis> | also I could see something weird now that I cant update any package using yum |
01:14 | <fayis> | it simply not connecting to repo |
01:14 | <fayis> | let me know what could be the reason ? |
01:15 | <millisa> | The output of iptables-save put into a pastebin would probably help |
01:15 | <millisa> | !paste |
01:15 | <linbot> | Please paste longer snippets over at https://bpaste.net/ and not in the channel |
01:15 | <millisa> | but generally if it pings, but you can't get through to the ports, it usually means you've got something incorrect with the firewall setup |
01:16 | <fayis> | sure I will get it done please hold on |
01:18 | <fayis> | https://bpaste.net/show/639e6c0ac222 |
01:18 | <fayis> | kindly check the output |
01:20 | <millisa> | you've cut off the top |
01:21 | <fayis> | oh let me check again |
01:22 | <fayis> | actually I cant scroll to top using their web console |
01:22 | <wraeth> | Can you connect out from the machine? |
01:22 | <fayis> | I cant connect only using the inbuilt console |
01:23 | <millisa> | https://www.linode.com/docs/platform/manager/using-the-linode-shell-lish/#use-a-terminal-application |
01:23 | <millisa> | or use a pager like less or more |
01:23 | <wraeth> | fayis: So the machine itself can't ping google or anything? |
01:23 | <millisa> | (i can ping their IP from another linode, so...) |
01:23 | <fayis> | @wraeth I think so |
01:24 | <wraeth> | If it can, `iptables-save | curl -F 'f:1=<-' ix.io` should paste the full output of iptables-save to ix.io and give you a paste URL. |
01:24 | <fayis> | @milisa thanks for the reference let me try to connect using it |
01:24 | <fayis> | ok let me try |
01:27 | <fayis> | iptables-save | curl -F 'f:1=<-' ix.io this command simply hangs |
01:27 | <fayis> | I think the machine cant access internet |
01:27 | <fayis> | :( |
01:27 | <wraeth> | Then try a terminal to lish as millisa suggested. |
01:27 | <fayis> | curl: (6) Could not resolve host: ix.io; Unknown error |
01:27 | <fayis> | ok checking |
01:31 | <fayis> | I tried using putty and got connected |
01:31 | <fayis> | executed the cmmand |
01:31 | <fayis> | https://bpaste.net/show/ca0f4e8e1d1d |
01:32 | <millisa> | that's still only the tail end. and it's odd that it looks different than what you had earlier. |
01:32 | <fayis> | yes me too thinks the same it is different |
01:33 | <fayis> | but I had done some enable/disable in firewalld |
01:33 | <fayis> | would it cause such an issue ? |
01:34 | <fayis> | should I try disabling the dome9 agent fully |
01:34 | <fayis> | and let the firewalld alone ? |
01:34 | <millisa> | probably. |
01:35 | <fayis> | okay let me uninstall the dome9 |
01:36 | <millisa> | (I can't ping your IP anymore, so who knows) |
01:38 | <wraeth> | Changing the thing someone is helping you investigate without noting that you're changing it, let alone /how/ you're changing it, doesn't make the investigation any easier... |
01:39 | <fayis> | could you give me a better advice then |
01:39 | <fayis> | I'm trying to uninstall the agent |
01:39 | <fayis> | and will send you the updated iptables |
01:39 | <fayis> | please hold on |
01:44 | <fayis> | https://bpaste.net/show/c3bc6d95ef04 |
01:44 | <fayis> | please check now |
01:44 | <fayis> | I'm extremely sorry for the delay |
01:45 | <millisa> | you would need to post more than the end of the output |
01:46 | <fayis> | sorry ? could you explain please |
01:46 | <millisa> | that isn't the full output of the command. |
01:46 | <fayis> | any other command to get the full output ? |
01:47 | <fayis> | I just tried ping to google from the console and it is not happening |
01:47 | <fayis> | not connecting |
01:49 | <millisa> | iptables-save usually starts with a line that says something like '# Generated by iptables-save ...' and ends with '# Completed on ...' |
01:50 | -!- | thiras [~thiras@94.122.154.44] has joined #linode |
01:50 | -!- | thiras is "Ant" on #tami #linode #debian |
01:50 | <fayis> | but I can only able to copy this lines :( |
01:51 | <fayis> | also I wonder why it keep changing |
01:51 | <fayis> | right now the rules seems to be changed again |
01:55 | <millisa> | You can page output of a command by using |less or |more |
01:55 | <millisa> | Something like: iptables-save | less |
01:56 | <millisa> | And honestly, if you have to be told that, you probably need to be hiring someone to administer your server |
02:00 | <fayis> | as having issue to connect to internet from within server ? |
02:00 | <fayis> | https://bpaste.net/show/c9378c8778a5 |
02:00 | <fayis> | please check the output |
02:01 | <millisa> | that looks like you stopped all your firewalls. and not surprisingly, i can ping your IP |
02:02 | <fayis> | omg :( |
02:03 | <fayis> | what to do now |
02:03 | <fayis> | but I had started firewalld :( |
02:03 | <fayis> | please help me out :/ |
02:04 | <millisa> | if that is your current output of iptables-save - can you ping 8.8.8.8 from the linode? |
02:05 | <fayis> | let me try |
02:06 | <fayis> | yes I can ping |
02:06 | <fayis> | --- 8.8.8.8 ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 4003ms rtt min/avg/max/mdev = 1.041/1.189/1.289/0.100 ms |
02:13 | <millisa> | I have never tried it, but supposedly there is a script in a default cpanel install at /usr/local/cpanel/scripts/configure_firewall_for_cpanel that will create the default port openings |
02:14 | <millisa> | it's mentioned down at the bottom of https://documentation.cpanel.net/display/CKB/How+to+Configure+Your+Firewall+for+cPanel+Services#HowtoConfigureYourFirewallforcPanelServices-cent7CentOS7,CloudLinux7,andRHEL7firewallmanagement |
02:17 | <fayis> | ok thanks let me try |
02:22 | <fayis> | done and now I can see the dome9 entries are gone |
02:22 | <fayis> | but still cant connect |
02:24 | <millisa> | you ran the configure_firewall_for_cpanel script? your iptables-save output did not change from the bare one you had before? or does it have entries for the port openings you need? |
02:28 | <fayis> | hello I had opened a ticket and they said they restricted the network |
02:28 | <fayis> | :( because of some phishing attempts from my client's hosted websites |
02:29 | <fayis> | thank you very much for your time and I appreciate your great support |
04:04 | -!- | Kantha [~oftc-webi@103.199.145.97] has joined #linode |
04:04 | -!- | Kantha is "OFTC WebIRC Client" on #linode |
04:04 | <Kantha> | Hi |
04:05 | <Kantha> | is anyone there? |
04:05 | <linbot> | Kantha: If you have a question, feel free to just ask it -- someone's always willing to help. If you don't get a response right away, be patient! You may want to read http://alexfornuto.com/how-to-ask-for-help-on-irc/ |
04:05 | <Kantha> | i need an SSL certificate for my linode server |
04:06 | <Kantha> | can you guys suggest about this ? |
04:07 | <wraeth> | I just use LetsEncrypt. |
04:14 | <Kantha> | Yeah i knew that and am already using that one |
04:15 | <Kantha> | But i need to set up as like cost wise ssl for period of years |
04:15 | <Woet> | Kantha: why? |
04:15 | <dwfreed> | let's encrypt typically autorenews every 60 days, and will (most likely) never cost anything |
04:16 | <Kantha> | is my site secure through the letsencrypt ssl? |
04:16 | <Kantha> | for a long period of time |
04:17 | <dwfreed> | no more or less secure than paying for a 1/2/3 year cert |
04:18 | <Kantha> | do you have any plan for paid ssl? |
04:18 | <dwfreed> | why? |
04:18 | <dwfreed> | paying money doesn't make it more secure |
04:19 | <wraeth> | LE certs are just as good as paid certs, and for much cheaper. |
04:19 | <dwfreed> | you might just as well light your money on fire for all the benefit paying for a cert gives you over using letsencrypt |
04:20 | <Kantha> | okay i can understand dwfreed |
04:21 | <Kantha> | so LE is enough for secure the site? right? |
04:21 | <dwfreed> | yes |
04:22 | <Kantha> | is autorenew one right? |
04:24 | <nate> | Kantha: There's really no difference between LE and other DV level certificates as far as validity goes with exception of LE's have a shorter lifespan than paid ones generally do, however LE has the capacity for local auto renewal which makes that not a huge deal |
04:25 | <Kantha> | okay Nate |
04:25 | <Kantha> | Thank You |
04:26 | -!- | fayis [~oftc-webi@103.65.33.255] has quit [Quit: Page closed] |
04:28 | -!- | Kantha [~oftc-webi@103.199.145.97] has quit [Quit: Page closed] |
04:28 | <Woet> | dwfreed: but what about the $10 million USD insurance |
04:28 | <Woet> | they pay it out all the time, right |
05:31 | -!- | gideon [~oftc-webi@41.66.239.19] has joined #linode |
05:31 | -!- | gideon is "OFTC WebIRC Client" on #linode |
05:33 | -!- | gideon [~oftc-webi@41.66.239.19] has quit [Remote host closed the connection] |
05:39 | -!- | gideon [~oftc-webi@41.66.225.215] has joined #linode |
05:39 | -!- | gideon is "OFTC WebIRC Client" on #linode |
05:41 | -!- | gideon [~oftc-webi@41.66.225.215] has quit [] |
06:37 | -!- | thiras [~thiras@94.122.154.44] has quit [Ping timeout: 480 seconds] |
06:40 | -!- | wcpan [~quassel@2400:8902::f03c:91ff:fee0:f952] has quit [Remote host closed the connection] |
06:44 | -!- | wcpan [~quassel@2400:8902::f03c:91ff:fee0:f952] has joined #linode |
06:44 | -!- | wcpan is "wcpan" on #linode #dot |
06:46 | -!- | thiras [~thiras@176.54.197.242] has joined #linode |
06:46 | -!- | thiras is "Ant" on #debian #linode #tami |
06:49 | -!- | wcpan [~quassel@2400:8902::f03c:91ff:fee0:f952] has quit [Remote host closed the connection] |
06:58 | -!- | wcpan [~quassel@2400:8902::f03c:91ff:fee0:f952] has joined #linode |
06:58 | -!- | wcpan is "wcpan" on #dot #linode |
06:58 | -!- | wcpan [~quassel@2400:8902::f03c:91ff:fee0:f952] has quit [Remote host closed the connection] |
06:59 | -!- | wcpan [~quassel@2400:8902::f03c:91ff:fee0:f952] has joined #linode |
06:59 | -!- | wcpan is "wcpan" on #dot #linode |
07:01 | -!- | thiras [~thiras@176.54.197.242] has quit [Ping timeout: 480 seconds] |
07:07 | <linbot> | New news from community: How to I disable execution of malicious .php files in WordPress upload directories? <https://www.linode.com/community/questions/17861> |
07:10 | -!- | thiras [~thiras@94.122.159.14] has joined #linode |
07:10 | -!- | thiras is "Ant" on #debian #linode #tami |
07:49 | -!- | arooni [~arooni__@li1062-244.members.linode.com] has joined #linode |
07:49 | -!- | arooni is "Arooni ZNC" on #linode #clay |
07:50 | -!- | arooni_team_b [~arooni__@li1062-244.members.linode.com] has quit [Ping timeout: 480 seconds] |
09:06 | -!- | anomie [~anomie@00018802.user.oftc.net] has joined #linode |
09:06 | -!- | anomie is "Anomie" on #linode |
09:17 | <linbot> | New news from community: How do I find the IP address for my domain? <https://www.linode.com/community/questions/17863> || How do I find the IP address for my domain? <https://www.linode.com/community/questions/17862> |
09:21 | -!- | logan [~logan@irc.protiumit.com] has quit [Remote host closed the connection] |
09:21 | -!- | logan [~logan@irc.protiumit.com] has joined #linode |
09:21 | -!- | logan is "logan" on #linode #ceph-ansible #ceph |
09:25 | -!- | Ankur [~oftc-webi@123.201.68.180] has joined #linode |
09:25 | -!- | Ankur is "OFTC WebIRC Client" on #linode |
09:25 | <Ankur> | Hello, Linode provides server in Canada region? |
09:26 | <@bbigger> | Ankur: We're aiming to release a Toronto data center on March 18, 2019 |
09:27 | <@bbigger> | In the meantime you can see our data center locations and test latency here: https://www.linode.com/speedtest |
09:32 | <Peng> | :O |
09:41 | <DrJ> | bbigger: didn't know that |
09:41 | <DrJ> | :) |
09:41 | <DrJ> | any plans for a north korea datacenter? |
09:42 | <@bbigger> | not after that summit... |
09:44 | <DrJ> | :) |
09:44 | -!- | blaboon [~blaboon@00026ecf.user.oftc.net] has joined #linode |
09:44 | -!- | mode/#linode [+o blaboon] by ChanServ |
09:44 | -!- | blaboon is "Bradley LaBoon" on @#linode |
10:00 | -!- | renato [~oftc-webi@189.50.6.150] has joined #linode |
10:00 | -!- | renato is "OFTC WebIRC Client" on #linode |
10:01 | -!- | renato [~oftc-webi@189.50.6.150] has quit [] |
10:07 | <linbot> | New news from community: Access NodeBalancer log files for originating IP <https://www.linode.com/community/questions/17864> |
10:20 | -!- | Ankur [~oftc-webi@123.201.68.180] has quit [Quit: Page closed] |
10:36 | -!- | Ryon [~rohara@0001aba5.user.oftc.net] has quit [Quit: Oops] |
10:38 | -!- | Ryon [~rohara@0001aba5.user.oftc.net] has joined #linode |
10:38 | -!- | Ryon is "Ryan" on #linode |
11:17 | -!- | joecool_ [~joecool@2601:8a:500:f00:75cd:ffc4:b423:5b1] has joined #linode |
11:17 | -!- | joecool_ is "Joe" on #linode #ck |
11:17 | <v0lksman> | newark issues again? |
11:21 | -!- | nate [NBishop@00013625.user.oftc.net] has quit [Ping timeout: 480 seconds] |
11:22 | -!- | joecool|mobile [~joecool@c-174-57-44-238.hsd1.nj.comcast.net] has quit [Ping timeout: 480 seconds] |
11:24 | -!- | thiras [~thiras@94.122.159.14] has quit [Ping timeout: 480 seconds] |
11:27 | <tafa2> | I get too many e-mail notifications from servers - is anyone using a log management tool they would recommend? |
11:27 | <tafa2> | Maybe something I can pipe e-mails into as well? |
11:38 | -!- | kaare_ [~kaare@ip-5-186-247-169.dhcp.fibianet.dk] has quit [Remote host closed the connection] |
11:40 | -!- | renato [~oftc-webi@189.50.6.150] has joined #linode |
11:40 | -!- | renato is "OFTC WebIRC Client" on #linode |
11:41 | <renato> | good afternoon |
11:41 | <renato> | i have problem |
11:41 | <renato> | could anybody help me? |
11:42 | <DrJ> | depends on what your problem is |
11:42 | -!- | renato_ [~renato@189.50.6.150] has joined #linode |
11:42 | -!- | renato_ is "realname" on #linode |
11:42 | <v0lksman> | tafa2: checkout either rollbar or datadog but it's dependent on your app type really |
11:42 | <renato_> | hello! |
11:42 | <DrJ> | hi |
11:42 | <renato_> | i have pboblem with linode |
11:43 | <DrJ> | k |
11:43 | <DrJ> | gonna need more than that |
11:43 | <renato_> | I can not log into the dashboard |
11:43 | -!- | renato [~oftc-webi@189.50.6.150] has quit [] |
11:44 | <renato_> | i dont understand you |
11:45 | <renato_> | could you help me? |
11:45 | <DrJ> | what dashboard are you talking about? |
11:45 | <DrJ> | the linode manager? |
11:46 | <renato_> | I am talking of the linode login |
11:46 | <renato_> | this message is show to me: The Beta Manager and API V4 are not available for legacy pre-pay accounts. Please convert to Hourly billing. |
11:47 | <renato_> | i do not know what to do |
11:48 | <smallclone> | renato_: you need to convert to hourly billing |
11:48 | <smallclone> | https://www.linode.com/docs/platform/billing-and-support/upgrade-to-hourly-billing/ |
11:48 | <smallclone> | if you have one of those plans with the prepaid annual discount, you will lose the discount |
11:48 | <smallclone> | only real caveat to be aware of, otherwise the cost is the same |
11:48 | <renato_> | ok |
11:49 | <renato_> | i understand |
11:49 | <renato_> | thank you |
11:49 | <smallclone> | sure |
11:49 | -!- | renato_ [~renato@189.50.6.150] has quit [Quit: Leaving] |
11:50 | -!- | kaare_ [~kaare@ip-5-186-247-169.dhcp.fibianet.dk] has joined #linode |
11:50 | -!- | kaare_ is "Kaare Rasmussen" on #linode |
11:55 | <DrJ> | welcome |
11:55 | <DrJ> | opps, wrong channel |
11:58 | <linbot> | New news from community: CNAME record not showing up after TTL period <https://www.linode.com/community/questions/17865> |
12:04 | -!- | nate [NBishop@207-255-41-254-dhcp.jst.pa.atlanticbb.net] has joined #linode |
12:04 | -!- | nate is "Nathan" on #linode #php |
12:18 | <linbot> | New news from community: My Linode was hosting a phishing site, why was it shut off after only 4 hours? <https://www.linode.com/community/questions/17866> |
12:19 | <millisa> | "There was a linode hosting a phishing site, why did it take four whole hours to shut them off?" |
12:20 | <millisa> | "Why did the police stop me after I only killed 3 people?!" |
13:13 | -!- | hdb2 [~josh@0001a4b4.user.oftc.net] has joined #linode |
13:13 | -!- | hdb2 is "Josh Lawrence" on #linode |
13:15 | -!- | thiras [~thiras@195.174.215.70] has joined #linode |
13:15 | -!- | thiras is "Ant" on #debian #linode #tami |
13:15 | -!- | hdb2 [~josh@0001a4b4.user.oftc.net] has quit [] |
14:03 | -!- | elliot007 [elliot007@suchznc.net] has quit [Ping timeout: 480 seconds] |
14:05 | <tafa2> | v0lksman thanks I'll check them out |
14:12 | -!- | NinetalesStarlight [~Lunawolf@c-73-11-11-6.hsd1.or.comcast.net] has joined #linode |
14:12 | -!- | NinetalesStarlight is "realname" on #linode |
14:18 | <linbot> | New news from community: Can I run a StackScript after creating a Linode? <https://www.linode.com/community/questions/17867> |
14:22 | -!- | Sheila [~oftc-webi@190.43.167.169] has joined #linode |
14:22 | -!- | Sheila is "OFTC WebIRC Client" on #linode |
14:31 | -!- | CodeMouse92 [~JasonMc92@00025241.user.oftc.net] has joined #linode |
14:31 | -!- | CodeMouse92 is "Jason C. McDonald" on #packaging #linode #c++ |
14:42 | -!- | Sheila [~oftc-webi@190.43.167.169] has quit [Quit: Page closed] |
14:46 | -!- | elliot007 [elliot007@suchznc.net] has joined #linode |
14:46 | -!- | elliot007 is "Ankit R Gadiya" on #linode #debian-vim #debian-boinc |
15:59 | -!- | graydon [~oftc-webi@cpe-70-121-160-55.satx.res.rr.com] has joined #linode |
15:59 | -!- | graydon is "OFTC WebIRC Client" on #linode |
15:59 | <graydon> | Hi everyone. new here. Are there any major rules I should be aware of? |
16:02 | <graydon> | I have a strange issue with one of my linode servers and my google searches are failing me |
16:02 | <linbot> | New news from status: Scheduled Network Maintenance - London <https://status.linode.com/incidents/458nj2zsk4r9> |
16:03 | <Peng> | What kind of issue? |
16:06 | <graydon> | It's EXTREMELY slow |
16:06 | <graydon> | if I ssh, it takes several seconds for it to log in, and then even basic commands take a long time to execute, all with a 2-3 second latency when typing |
16:06 | <graydon> | It's new, as of yesterday when it started slow and go worse and worse and worse over the whole day |
16:16 | <graydon> | Another IRC channel I'm in suggested I open a ticket with Linode (which I have done), but I was hoping for any ideas while I wait |
16:20 | <AlexMax> | graydon: I'm not Linode, but which datacenter? |
16:22 | <graydon> | Newark |
16:22 | <graydon> | but the linode status seems to indicate no problems, and I have a few other servers at that datacenter which are fine |
16:24 | <Peng> | Can you determine if it's a general networking thing or a Your Linode thing? |
16:24 | <Peng> | E.g. use mtr and do stuff with a different Linode, e.g. the main lish console? |
16:24 | <Peng> | And check htop/top/iostat/vmstat/whatever for things like CPU usage, steal, context switches, disk latency...? |
16:25 | <graydon> | OK, so mtr from the slow server to 8.8.8.8 returned concerning, very high latency results. Same command from a different server showed no such problems. Same command from my machine also showed no problems |
16:25 | <graydon> | Cpu usage is low (as reported by linode, and the top command) |
16:27 | <graydon> | results from iostat: |
16:28 | <graydon> | https://www.screencast.com/t/iftXUpGwdF1P |
16:28 | <Peng> | Those CPU numbers are alarming |
16:28 | <linbot> | New news from community: mysql not restarting after server reboot <https://www.linode.com/community/questions/17868> |
16:29 | <Peng> | If you use "iostat -x", it also shows numbers about disk latency |
16:29 | <graydon> | iostat -x: https://www.screencast.com/t/StMXBFj3jwJg |
16:29 | <graydon> | i'm very much a noob at this. Not sure what I'm looking at |
16:30 | <Peng> | 51% steal is very bad and probably what's making everything feel slow. |
16:31 | <Peng> | Problematically high steal is something you should contact support about, by the way. |
16:31 | <Peng> | 30% sys time is odd. Might be part and parcel with the other stuff, or might indicate you're doing something odd, I dunno. |
16:31 | <graydon> | Oh ok, great. Good to know. Thanks for the tip on steal |
16:33 | <graydon> | If it helps at all, all this started (or at least I noticed it) when I was setting up new nginx sites on the server through laravel forge. I noticed it took a long time for it to open up the nginx config files there. |
16:33 | -!- | thiras [~thiras@195.174.215.70] has quit [Ping timeout: 480 seconds] |
16:33 | <graydon> | No idea if that's relevant or not |
16:33 | <Peng> | The disk I/O latency is also high, but that's probably caused by the CPU issue. And disk latency doesn't harm stuff that doesn't touch the disk. |
16:34 | <graydon> | OK, so while I wait for a response to my linode ticket |
16:34 | <Peng> | Steal means your VPS wants to use the CPU, but has to wait in line because other customers are using the CPU and there's not enough to go around. |
16:34 | <graydon> | what could I try in the meantime? Would it help to upgrade to a larger server? (because more resources, or because different vm)? |
16:35 | <Peng> | If *you're* the one using too much CPU, it means you're being a jerk and should stop. :P But in this case that's not you. |
16:35 | <dwfreed> | resizing would move you to a different host, which may have more or less resource contention |
16:35 | <Peng> | Hopefully not more! |
16:35 | <graydon> | I see. |
16:35 | <graydon> | hm. |
16:36 | <graydon> | Looked at a few other systems and they have like, 0 steal |
16:36 | <graydon> | so that's interesting. |
16:36 | <graydon> | Although I want to go back a step and ask about something |
16:36 | <Peng> | Yeah |
16:37 | <graydon> | You said "if you're the one using too much CPU" |
16:37 | <graydon> | I don't *think* I am using too much cpu... what's the best way to check that in this context? As I mentioned, linode is reporting like... 4-8% cpu usage |
16:37 | <graydon> | and this is a live server, like... it's hosting a site that people visit regularly |
16:38 | <graydon> | (so I expect some, obviously) |
16:38 | <Peng> | The user and nice % add up to barely 0.1%, so it probably has nothing to do with you. |
16:38 | <graydon> | interesting |
16:38 | <graydon> | OK. I will try upgrading to bigger linode (and maybe going back down, depending) while I wait for linode to review my ticket |
16:42 | <Peng> | Anyone have thoughts on those iostat screenshots? 0.1% user and nice, 30% sys, 51% steal. 100 ms disk latency. <1 GB disk I/O. |
16:42 | <Peng> | Is the sys time and disk latency *just* a symptom of the steal, or could some malfunctioning kernel module be eating time? |
16:43 | <Peng> | Or some workload like updatedb that does huge disk I/O and little CPU |
16:53 | <graydon> | Just heard back from Linode support. They also pointed out resource contention on the server. they're migrating me now. We'll see! |
16:53 | <graydon> | Thanks for your help Peng. I am very much still interested if anyone has answers to Peng's questions above ^^ |
16:57 | -!- | anomie [~anomie@00018802.user.oftc.net] has quit [Remote host closed the connection] |
17:29 | -!- | anew [~anew@107.red-83-46-234.dynamicip.rima-tde.net] has joined #linode |
17:29 | -!- | anew is "realname" on #linode |
18:04 | -!- | blaboon [~blaboon@00026ecf.user.oftc.net] has quit [Remote host closed the connection] |
18:07 | -!- | jarryd [jarryd@im.jarryd.net] has quit [Quit: jarryd] |
18:09 | -!- | anew [~anew@107.red-83-46-234.dynamicip.rima-tde.net] has quit [Read error: Connection reset by peer] |
18:09 | -!- | jarryd [jarryd@im.jarryd.net] has joined #linode |
18:09 | -!- | jarryd is "jarryd" on #linode |
18:29 | <nate> | Random question; anyone come up with a guide script for doing let's encrypt automatic renewals w/ linode's DNS API? or am I gonna have to rig something up myself |
18:30 | <Peng> | Some ACME clients come with support for it. |
18:30 | <Peng> | The 15+ minute delay is kind of painful though. |
18:32 | <nate> | Ah yeah didn't think of that, ugh |
18:33 | <nate> | maybe I should just throw their domain on my own personal nameservers and write a hook there... though that might be a bit more time than I want to put into this |
19:06 | -!- | KindOne [kindone@kindone.user.oftc.net] has quit [Ping timeout: 480 seconds] |
19:12 | -!- | KindOne [kindone@kindone.user.oftc.net] has joined #linode |
19:12 | -!- | KindOne is "..." on #tor-dev #suckless #qemu #php #ovirt #oftc #moocows #linode #libevent #https-everywhere #gentoo #g7 #freenode #eff #debian-next #debian #ceph #bcache |
19:13 | <millisa> | https://certbot-dns-linode.readthedocs.io/en/stable/ (they do have a mention at the bottom about having a 1000 second delay to account for it) |
19:13 | <millisa> | haven't gotten to try it yet |
19:17 | <SleePy> | Oh nice.. Would make it easier to issue *.domain.tld and *.*.domain.tld for my sites. Instead of a massive list of domains on a single cert. |
19:17 | <Peng> | *.*.domain.tld is impossible |
19:17 | -!- | graydon [~oftc-webi@cpe-70-121-160-55.satx.res.rr.com] has quit [Quit: Page closed] |
19:18 | <SleePy> | I thought certbot allowed *.*.doamin.tld :( |
19:18 | <Peng> | CAs don't. |
19:18 | <SleePy> | But it lets *.sub.domain.tld though? |
19:18 | <Peng> | Yes |
19:19 | <SleePy> | Perfect. What I meant to do anyways |
19:20 | <nate> | I've seen some that do with a warning that support may not always be there (ie; Internet Explorer), but that was ages ago |
19:47 | <nate> | Welp tried to do certbot with the linode dns plugin and apparently completely broke certbot lol |
19:49 | <nate> | Ah nevermind looks like the linode dns plugin needs a far newer certbot than the extended ubuntu repos have, lame |
19:51 | <Peng> | The Certbot plugin's not easy to install on Ubuntu. :( Except for Cosmic and Disco. |
19:53 | <nate> | installing the plugin wasn't hard at all, found it in pip, it's just it expects certbot >= 0.33 and ubuntu apparently has 0.28 :/ |
19:54 | * | ajmitch_ prefers acme.sh, easier to manage |
19:54 | <nate> | well of acme.sh has a method for easily accessing linode's DNS API for wildcard certs, I'll gladly look at it :P |
19:55 | <Peng> | It does |
19:57 | -!- | kwatson [~kwatson@ns532107.ip-198-100-144.net] has quit [Remote host closed the connection] |
20:00 | <nate> | -easily- accessing? I shall take a look then when I get back :P |
20:07 | <dwfreed> | I run my own hidden master anyway, so I just point dehydrated's dnsupdate hook at that |
20:18 | -!- | Geezus42[m] [~geezus42m@2001:470:1af1:101::2ff] has joined #linode |
20:18 | -!- | Geezus42[m] is "@Geezus42:matrix.org" on #linode #alpine-linux #mm |
20:30 | -!- | millisa_ [~millisa@rrcs-71-42-149-155.sw.biz.rr.com] has joined #linode |
20:30 | -!- | millisa_ is "realname" on #linode |
21:03 | -!- | rainbow [~ssmith@00020809.user.oftc.net] has joined #linode |
21:03 | -!- | rainbow is "Samantha "Rainbow" Smith" on #linode-tavern #linode |
21:08 | -!- | RainbowLin [~ssmith@caskmaker.hacker.horse] has quit [Ping timeout: 480 seconds] |
21:29 | <linbot> | New news from community: How can connect to SSH ? <https://www.linode.com/community/questions/17869> |
22:00 | <linbot> | New news from community: How can I connect to SSH ? <https://www.linode.com/community/questions/17869> |
22:08 | -!- | millisa_ [~millisa@rrcs-71-42-149-155.sw.biz.rr.com] has quit [Quit: Leaving] |
22:26 | -!- | CodeMouse92 [~JasonMc92@00025241.user.oftc.net] has quit [Quit: Oh freddled gruntbuggly | Thy micturations are to me | As plurdled gabbleblotchits | On a lurgid bee] |
23:19 | -!- | joey [~oftc-webi@175.136.1.105] has joined #linode |
23:19 | -!- | joey is "OFTC WebIRC Client" on #linode |
23:19 | <joey> | Hi, has anyone ever been locked out of their account due to 2FA before? |
23:19 | <millisa> | sure, use your scratch code |
23:20 | <joey> | I did not record the scratch code as I didn't think this would happen :-( |
23:21 | <millisa> | next time, record the scratch code. you'll need to contact them if you dont have it https://www.linode.com/docs/platform/manager/keep-your-linode-account-safe/#recovery-procedure |
23:21 | <joey> | I've contacted Linode twice with the pictures they requested but have received no reply. I was wondering if there are any tech support people in this channel |
23:26 | <rsdehart> | !ops |
23:26 | <linbot> | Users with ops are employees of Linode, and know what they're talking about. The rest of us are the ever-so-helpful(?) community. Official Linode contact information: https://www.linode.com/contact |
23:26 | <rsdehart> | joey: ^ |
23:27 | <joey> | Thanks. Guess I'll have to give them a call |
23:29 | -!- | joey [~oftc-webi@175.136.1.105] has quit [Quit: Page closed] |
--- | Log | closed Fri Mar 01 00:00:24 2019 |