Back to Home / #linode / 2019 / 02 / Prev Day | Next Day
#linode IRC Logs for 2019-02-05

---Logopened Tue Feb 05 00:00:51 2019
00:12-!-bumbleVole [~bumblevol@pool-108-41-237-196.nycmny.fios.verizon.net] has joined #linode
00:12-!-bumbleVole is "bumbleVole" on #linode
00:20-!-bumbleVole [~bumblevol@pool-108-41-237-196.nycmny.fios.verizon.net] has quit [Ping timeout: 480 seconds]
02:02-!-bumbleVole [~bumblevol@pool-108-41-237-196.nycmny.fios.verizon.net] has joined #linode
02:02-!-bumbleVole is "bumbleVole" on #linode
02:04<linbot>New news from community: How to update ACME client for Letsencrypt new validation method? <https://www.linode.com/community/questions/17714>
02:10-!-bumbleVole [~bumblevol@pool-108-41-237-196.nycmny.fios.verizon.net] has quit [Ping timeout: 480 seconds]
02:20<Peng_>!web title https://community.letsencrypt.org/t/how-to-update-letsencrypt-acme-client-for-new-validation-method/85275
02:20<linbot>Peng_: An error has occurred and has been logged. Please contact this bot's administrator for more information.
02:20<Peng_>Wow
02:20<Peng_>I'm getting deja vu. But not about linbot.
02:44<linbot>New news from community: Is Object Storage Going to Ever Happen? <https://www.linode.com/community/questions/17715>
02:51-!-bumbleVole [~bumblevol@pool-108-41-237-196.nycmny.fios.verizon.net] has joined #linode
02:51-!-bumbleVole is "bumbleVole" on #linode
02:59-!-bumbleVole [~bumblevol@pool-108-41-237-196.nycmny.fios.verizon.net] has quit [Ping timeout: 480 seconds]
03:27-!-kyhwana_ [~kyhwana@103.247.154.59] has joined #linode
03:27-!-kyhwana_ is "kyhwana" on #linode
03:27-!-kyhwana [~kyhwana@103.247.154.59] has quit [Read error: Connection reset by peer]
04:01-!-bumbleVole [~bumblevol@pool-108-41-237-196.nycmny.fios.verizon.net] has joined #linode
04:01-!-bumbleVole is "bumbleVole" on #linode
04:02-!-thiras [~thiras@195.174.215.70] has joined #linode
04:02-!-thiras is "Ant" on #debian #linode #tami
04:09-!-bumbleVole [~bumblevol@pool-108-41-237-196.nycmny.fios.verizon.net] has quit [Ping timeout: 480 seconds]
04:22-!-mw_ [~oftc-webi@cpc81415-lanc8-2-0-cust134.3-3.cable.virginm.net] has joined #linode
04:22-!-mw_ is "OFTC WebIRC Client" on #linode
04:22-!-mw_ [~oftc-webi@cpc81415-lanc8-2-0-cust134.3-3.cable.virginm.net] has quit []
04:29-!-thiras [~thiras@195.174.215.70] has quit [Ping timeout: 480 seconds]
05:01-!-help [~oftc-webi@116.68.114.143] has joined #linode
05:01-!-help is "OFTC WebIRC Client" on #linode
05:02-!-help is now known as Guest306
05:02<Guest306>I am getting POST request from Somebody ... (an ATTACK!) . i blocked blank refer.. but still they are sending request.. server is retuning 403.. but incomming traffic is over 500Mbps
05:02<Guest306>72.240.18.224 - - [05/Feb/2019:09:38:51 +0000] "POST /upload HTTP/1.1" 403 134 "-" "2106252114" 24.47.120.162 - - [05/Feb/2019:09:38:51 +0000] "POST /upload HTTP/1.1" 403 134 "-" "-1081571969" 74.197.143.242 - - [05/Feb/2019:09:38:51 +0000] "POST /upload HTTP/1.1" 403 134 "-" "-1682679847" 24.136.31.9 - - [05/Feb/2019:09:38:51 +0000] "POST /upload HTTP/1.1" 403 134 "-" "-509808108"
05:02<Guest306>how to prevent this? i am using Nginx
05:07<nate>That's not really something you can 'prevent' if they're literally sending 500Mbit traffic at you, you could simply 404 any attempt to request /upload but maybe you should look into something like cloudflare for a time
05:08<Guest306>if ($http_referer = "") { return 403; }
05:09<Guest306>i understand .. changing to 404 will help without moving to cloudflare ?
05:14<Woet>Guest306: that's a silly rule, many legitimate visitors won't have a referrer.
05:15<Woet>Guest306: you can't prevent incoming traffic, Linode doesn't offer a firewall at network level
05:16<Guest306>@Woet. This is not the main server. All i care is the requests from my main domain. this server is specifically for processing data
05:16<Woet>Guest306: yea, and there's plenty of browsers and extensions that strip referrer
05:16<Woet>Guest306: why don't you want to use Cloudflare?
05:17<Guest306>normal users should not need to access this server without proper referrer
05:17<Woet>Guest306: there's plenty of normal users using browsers and extensions that strip referrer
05:17<Guest306>because it's slow and i run a lot of custom caching setup
05:18<Woet>Guest306: Cloudflare is not slow.
05:18<Guest306>oh.. i got it.. OMG
05:18<Woet>quite the opposite in the majority of cases
05:19<Guest306>Yes, i have tested them. And if there is no other solution i should move to cloudflare :(
05:21<Guest306>Incoming: 24275 GB for a single core server LOL
05:21<Guest306>in 4 days :(
05:23<Guest306>GOOD NEWS!!! changing if ($http_referer = "") { return 403; } to if ($http_referer = "") { return 444; } stopped this S___t for NOW.. Thank God..
05:26<Guest306>Thanks bro for the response. now CPU usage under control .. everything fine :)
05:29<Woet>Guest306: that doesn't do anything to stop the requests.
05:31<Guest306>But it actually did. I think it is an Android or iOs App.. doing this to my server. all requests are from random IP without proper referrer
05:36<Guest306>444 Connection Closed Without Response. A non-standard status code used to instruct nginx to close the connection without sending a response to the client, most commonly used to deny malicious or malformed requests.
05:36<nate>Guest306: Cloudflare isn't something you "move" to, it's a service you can put in the middle that will try to automatically handle what they view as malicious traffic
05:37<Woet>Guest306: yes, and an attacker doesn't go "oh no, they're sending me 444 instead of 403s, better stop"
05:37<Woet>Guest306: does /upload actually work? what is it supposed to do?
05:38<Guest306>It is a distributed desktop or mobile program. So 444 currently stopped the attack. and my service is running fine.. Yes.. Nate i understand cloudflare will be better to stop attack like this..
05:39<Woet>Guest306: those IPs don't show up on any blacklists, so it's doubtful it's an attack.
05:40<Woet>Guest306: "It is a distributed desktop or mobile program" is not very descriptive
05:40<Guest306>@Wote. /Upload will handle file upload request from main domain/server ... a PHP script will detect new files and process it. BUT this attacker need to use my service without asking permission.. " Leaching ?" ..
05:42<Woet>Guest306: time to make it less enticing for people to (ab)use then
05:42<Guest306>we cannot "blacklists" . because it is completely random IP .. A desktop browser cannot do a cross domain request like this.. maybe an app or Extension
05:43<Woet>Guest306: I didn't say blacklist. what does /upload do and return as output?
05:46<Guest306>@Woet.. It will process the file and return "done" file can be accessed from another dir using the request Id.. i make it simple because i can add more servers and handle more requests. but attacker learned how i do this and making requests from his app or extension .. simply he dose't want to pay for the server :P
05:46<Woet>Guest306: yea, having an open uploader like that will be abused.
05:46<Woet>Guest306: add a captcha or similar tools.
05:46<Guest306>yes,, defiantly i need to do that..
05:55-!-bumbleVole [~bumblevol@pool-108-41-237-196.nycmny.fios.verizon.net] has joined #linode
05:55-!-bumbleVole is "bumbleVole" on #linode
05:59-!-Guest306 [~oftc-webi@116.68.114.143] has quit [Remote host closed the connection]
06:03-!-bumbleVole [~bumblevol@pool-108-41-237-196.nycmny.fios.verizon.net] has quit [Ping timeout: 480 seconds]
06:08<rsdehart>why does it always have to be defiantly? WHy can't someone do something amenably sometimes?
06:12<Woet>yea, it truly defies logic.
06:37-!-dueyfinster [~dueyfinst@51-171-118-84-dynamic.agg2.rsl.rsl-rtd.eircom.net] has quit [Remote host closed the connection]
06:38-!-dueyfinster [~dueyfinst@51-171-118-84-dynamic.agg2.rsl.rsl-rtd.eircom.net] has joined #linode
06:38-!-dueyfinster is "NG" on #linode
06:38-!-dueyfinster [~dueyfinst@51-171-118-84-dynamic.agg2.rsl.rsl-rtd.eircom.net] has quit [Remote host closed the connection]
06:40-!-dueyfinster [~dueyfinst@51-171-118-84-dynamic.agg2.rsl.rsl-rtd.eircom.net] has joined #linode
06:40-!-dueyfinster is "NG" on #linode
07:24-!-thiras [~thiras@195.174.215.70] has joined #linode
07:24-!-thiras is "Ant" on #debian #linode #tami
07:39-!-xyz [~oftc-webi@123.201.194.83] has joined #linode
07:39-!-xyz is "OFTC WebIRC Client" on #linode
07:39-!-xyz [~oftc-webi@123.201.194.83] has quit [Remote host closed the connection]
07:53-!-bumbleVole [~bumblevol@pool-108-41-237-196.nycmny.fios.verizon.net] has joined #linode
07:53-!-bumbleVole is "bumbleVole" on #linode
07:56-!-anomie [~anomie@00018802.user.oftc.net] has joined #linode
07:56-!-anomie is "Anomie" on #linode
08:01-!-bumbleVole [~bumblevol@pool-108-41-237-196.nycmny.fios.verizon.net] has quit [Ping timeout: 480 seconds]
09:05<linbot>New news from community: Multiple IP: Auto or Manual? <https://www.linode.com/community/questions/17716>
09:22-!-m_singh [~oftc-webi@2405:201:c802:7739:91a:10b4:be18:4c05] has joined #linode
09:22-!-m_singh is "OFTC WebIRC Client" on #linode
09:24<m_singh>Hi, I am looking for a small Linux machine, do Linode offer free trial for ~5-7 days ?
09:24<m_singh>so that my team can see the performance
09:32<dzho>no free trial, but there is a 7 day money back guarantee, or so I'm given to understand
09:32<dzho>so I guess the end result could be the same:
09:32<dzho>use it for 7 days and have paid nothing
09:32<dzho>Net
09:39-!-redentor [~redentor@189.202.73.238.cable.dyn.cableonline.com.mx] has joined #linode
09:39-!-redentor is "realname" on #debian-mx #debian-es #debian #linode
09:51-!-asilverman [~aaron@2600:3c03::f03c:91ff:fef0:6cce] has joined #linode
09:51-!-asilverman is "aaron" on #linode
09:52-!-asilverman [~aaron@2600:3c03::f03c:91ff:fef0:6cce] has quit []
10:12-!-m_singh [~oftc-webi@2405:201:c802:7739:91a:10b4:be18:4c05] has quit [Quit: Page closed]
10:13-!-uther [~uther@0002628b.user.oftc.net] has quit [Quit: hello darkness my old friend...]
10:15-!-bumbleVole [~bumblevol@pool-108-41-237-196.nycmny.fios.verizon.net] has joined #linode
10:15-!-bumbleVole is "bumbleVole" on #linode
10:20-!-uther [~uther@0002628b.user.oftc.net] has joined #linode
10:20-!-uther is "sexual salmon" on #linode
10:26-!-Quantum [~oftc-webi@34.63.151.186.static.intelnet.net.gt] has joined #linode
10:26-!-Quantum is "OFTC WebIRC Client" on #linode
10:26<Quantum>IDENTIFY
10:27<Quantum>hi
10:27<Quantum>I need help, can anybody help me?
10:28<synfinatic>!ask
10:28<linbot>If you have a question, feel free to just ask it -- someone's always willing to help. If you don't get a response right away, be patient! You may want to read http://alexfornuto.com/how-to-ask-for-help-on-irc/
10:31<Quantum>Ok, I've read the golden rules of IRC. Can anyone help me, my linode system has been compromised and, I have backups, is the retore from backup an option? If I rebuild, can I use the backup images to restore and what do I need to reconfigure?
10:32<synfinatic>well if your backup is prior to your getting compromised you can restore that. and hopefully also fix the vulnerability so you don't get compromised again
10:33<synfinatic>another option: launch a new instance and migrate your data (and making to sure to patch/configure your software more securely)
10:34<synfinatic>and of course: change your passwords
10:47<Quantum>Can I use my backups to migrate my data? of course, prior to the comprimise
11:16<synfinatic>https://www.linode.com/community/questions/6579/restore-backup-on-different-linode
11:16<synfinatic>tja
11:22-!-bumbleVole [~bumblevol@pool-108-41-237-196.nycmny.fios.verizon.net] has quit [Ping timeout: 480 seconds]
11:43-!-titan1018 [~oftc-webi@222.101.138.166] has joined #linode
11:43-!-titan1018 is "OFTC WebIRC Client" on #linode
11:43-!-titan1018 [~oftc-webi@222.101.138.166] has quit []
11:50-!-bumbleVole [~bumblevol@pool-108-41-237-196.nycmny.fios.verizon.net] has joined #linode
11:50-!-bumbleVole is "bumbleVole" on #linode
11:57-!-compuguy [~compuguy@ec2-34-193-165-89.compute-1.amazonaws.com] has quit [Quit: The Lounge - https://thelounge.github.io]
11:58-!-bumbleVole [~bumblevol@pool-108-41-237-196.nycmny.fios.verizon.net] has quit [Ping timeout: 480 seconds]
11:58-!-compuguy [~compuguy@ec2-34-193-165-89.compute-1.amazonaws.com] has joined #linode
11:58-!-compuguy is "compuguy" on #linode
12:00-!-compuguy [~compuguy@ec2-34-193-165-89.compute-1.amazonaws.com] has quit []
12:01-!-compuguy [~compuguy@ec2-34-193-165-89.compute-1.amazonaws.com] has joined #linode
12:01-!-compuguy is "compuguy" on #linode
12:05-!-compuguy62 [~compuguy@ec2-34-193-165-89.compute-1.amazonaws.com] has joined #linode
12:05-!-compuguy62 is "compuguy" on #linode
12:08-!-compuguy62 [~compuguy@ec2-34-193-165-89.compute-1.amazonaws.com] has quit []
12:11-!-compuguy [~compuguy@ec2-34-193-165-89.compute-1.amazonaws.com] has quit [Ping timeout: 480 seconds]
12:12-!-compuguy62 [~compuguy@ec2-34-193-165-89.compute-1.amazonaws.com] has joined #linode
12:12-!-compuguy62 is "compuguy" on #linode
12:17-!-compuguy62 is now known as compuguy
12:44-!-coast [~oftc-webi@2605:e000:1c01:c0b0:f886:bcd8:7f61:5da8] has joined #linode
12:44-!-coast is "OFTC WebIRC Client" on #linode
12:45<coast>noob
12:56<linbot>New news from community: Why won't Apache start? mktemp failed to create file via template <https://www.linode.com/community/questions/17717>
12:59-!-Strykar [~wakka@strykar.user.oftc.net] has quit [Quit: /quit]
13:00-!-Strykar [~wakka@strykar.user.oftc.net] has joined #linode
13:00-!-Strykar is "vector" on #linode #bitrig #bitlbee
13:06<linbot>New news from community: How do I register a website domain at Linode? <https://www.linode.com/community/questions/17718>
13:08-!-GFX [~oftc-webi@31.59.155.192] has joined #linode
13:08-!-GFX is "OFTC WebIRC Client" on #linode
13:09<GFX>hello
13:09<GFX>???????
13:09<millisa>hi
13:10<GFX>how are you
13:10<synfinatic>is your name elder price?
13:10<GFX>wtf
13:10<synfinatic>ah, you haven't seen the book of mormon
13:12-!-GFX [~oftc-webi@31.59.155.192] has quit []
13:12<millisa>i'm wagering it didn't play much in iran
13:13-!-redentor [~redentor@189.202.73.238.cable.dyn.cableonline.com.mx] has quit [Remote host closed the connection]
13:14<synfinatic>i'm glad i'm busy or one of these days i'd create an auto-responder to "hello" which randomly selects the BoM intro or Lionel Richie
13:29-!-ughguh [~oftc-webi@cpe-76-88-0-222.san.res.rr.com] has joined #linode
13:29-!-ughguh is "OFTC WebIRC Client" on #linode
13:30-!-ughguh [~oftc-webi@cpe-76-88-0-222.san.res.rr.com] has quit [Remote host closed the connection]
14:03-!-marshmn [~matt@84.93.31.67] has joined #linode
14:03-!-marshmn is "Matt Marsh" on #linode
14:04-!-KindOne [kindone@kindone.user.oftc.net] has quit [Ping timeout: 480 seconds]
14:06<linbot>New news from blog: Introducing: Linode Dedicated CPU Instances <https://blog.linode.com/2019/02/05/introducing-linode-dedicated-cpu-instances/>
14:07-!-ChanServ changed the topic of #linode to: Linode Community Discussion | https://www.linode.com/community | Introducing: Linode Dedicated CPU Instances - https://blog.linode.com/2019/02/05/introducing-linode-dedicated-cpu-instances/
14:07<millisa>finally, a factorio instance!
14:10<Peng_>:O
14:10<Peng_>It can handle so much NTP!
14:11-!-KindOne [kindone@kindone.user.oftc.net] has joined #linode
14:11-!-KindOne is "..." on #tor-dev #suckless #qemu #php #ovirt #oftc #moocows #linode #libevent #https-everywhere #gentoo #g7 #freenode @#eff #debian-next #debian #ceph #bcache
14:15<millisa>first one I tried had: 2x Intel(R) Xeon(R) CPU E5-2697 v4 @ 2.30GHz
14:17<millisa>(just spinning it up to try migrating to/from them)
14:19<csnxs>no ram, no disk, no network - only cpu!
14:19<@jhaas>dw, we give you network, otherwise you wouldn't be able to https://downloadmoreram.com/
14:22<csnxs>but can I https://downloadmorewam.com/
14:24<@jhaas>o_o; love random links that make my day more surreal
14:27<csnxs>( ded-otated WAAM )
14:27<millisa>is this that asmr thing I've been hearing about?
14:28<millisa>because, I got tingles
14:29-!-coast [~oftc-webi@2605:e000:1c01:c0b0:f886:bcd8:7f61:5da8] has quit [Quit: Page closed]
14:29<csnxs>that's an interesting category of asmr...
14:30<millisa>(migration from dedicated 2core -> nanode. works. also ended up on a e5-2697 v4 there)
14:32-!-marshmn [~matt@84.93.31.67] has quit [Ping timeout: 480 seconds]
14:35<csnxs>my preferred asmr is "todd howard seduces you in the bathroom at a fleetwood mac concert"
14:36<millisa>first criticism: Plan column on dashboard just says 'Linode 4096' in the old manager. New manager shows 'dedicated' to tell the difference
14:37-!-KindOne [kindone@kindone.user.oftc.net] has quit [Read error: Connection reset by peer]
14:38-!-Quantum [~oftc-webi@34.63.151.186.static.intelnet.net.gt] has quit [Quit: Page closed]
14:50<warewolf>oh no
14:51<warewolf>that fella who wanted to do openbroadcaster streaming is gonna come back now
15:00-!-SwadeNet [~oftc-webi@71-10-181-208.dhcp.stls.mo.charter.com] has joined #linode
15:00-!-SwadeNet is "OFTC WebIRC Client" on #linode
15:03-!-marshmn [~matt@84.93.31.67] has joined #linode
15:03-!-marshmn is "Matt Marsh" on #linode
15:03<staticsafe>I'm more excited about the planned Canadian location :o :o
15:04<SwadeNet>hello
15:04<millisa>greetings
15:05<warewolf>niihao
15:05<SwadeNet>so i need help with setting up a site if anyone knows how?
15:05<millisa>you'd have to be a bit more specific
15:06<SwadeNet>trying to install invision board board forums but seem to forgot how
15:07<SwadeNet>if anyone could help that would be great
15:07<warewolf>SwadeNet: are there any install docs?
15:07<millisa>again, you'd really have to be more specific.
15:07<SwadeNet>yes kinda hold on
15:09<SwadeNet>more specific? like what i need to get them installed etc?
15:09<SwadeNet>Self-hosting our software requires a server with these minimum capabilities: PHP 5.6+ (PHP 7 recommended) All default extensions GD2 extension mbstring extension MySQL 5.1+ (MySQL 5.6+ recommended) Our software works with all modern web servers (Apache, nginx, lighttpd, etc.) that can support the listed requirements. Please note that if you choose to use Windows based hosting, our technical support services will be limited.
15:10<SwadeNet>https://invisioncommunity.com/buy/self-hosted/
15:10<warewolf>so what webserver are you running?
15:11<SwadeNet>none atm need help installing it.
15:11<millisa>Did you look at any of these: https://www.linode.com/docs/web-servers/lamp/
15:12<SwadeNet>i did not see those im on debian 9 so im guessing i can use that
15:13<SwadeNet>if anyone can help me set it up ill pay
15:14<warewolf>SwadeNet: it'd be better if you paid for linode managed hosting, rather than throwing money at some random person you just met on IRC
15:15<SwadeNet>dont they charge like $100?
15:15<warewolf>I don't know what the expense is off hand
15:16<synfinatic>uh, so you are willing to pay, but $100 is too much? maybe you can find a high school student to do it for less, but any professional consultant will likely charge more
15:17<warewolf>SwadeNet: good news! You can pay invision to install it for you.
15:18<warewolf>" We offer initial installation as an optional service, priced at $100. " right on that page you linked earlier.
15:18<warewolf>just do that.
15:18<SwadeNet>No need to be rude my dude i have other expenses and thanks warewolf
15:23-!-bumbleVole [~bumblevol@pool-108-41-237-196.nycmny.fios.verizon.net] has joined #linode
15:23-!-bumbleVole is "bumbleVole" on #linode
15:24-!-SwadeNet [~oftc-webi@71-10-181-208.dhcp.stls.mo.charter.com] has quit [Quit: Page closed]
15:29-!-bumbleVole [~bumblevol@pool-108-41-237-196.nycmny.fios.verizon.net] has quit [Remote host closed the connection]
15:29-!-bumbleVole [~bumblevol@pool-108-41-237-196.nycmny.fios.verizon.net] has joined #linode
15:29-!-bumbleVole is "bumbleVole" on #linode
15:35-!-deetwelve [~deetwelve@00021c14.user.oftc.net] has quit [Quit: null]
15:47-!-deetwelve [~deetwelve@00021c14.user.oftc.net] has joined #linode
15:47-!-deetwelve is "deetwelve" on #oftc #tor-project #linode
15:54-!-shan [~oftc-webi@wsip-70-165-50-98.oc.oc.cox.net] has joined #linode
15:54-!-shan is "OFTC WebIRC Client" on #linode
15:54-!-shan [~oftc-webi@wsip-70-165-50-98.oc.oc.cox.net] has quit []
16:27-!-bumbleVole [~bumblevol@pool-108-41-237-196.nycmny.fios.verizon.net] has quit [Remote host closed the connection]
16:36-!-internat [biteme2@14-202-4-62.tpgi.com.au] has joined #linode
16:36-!-internat is "Nathan" on #tardigans @#binfenv #linode
16:42-!-scivola [~scivola@scivo.la] has quit [Quit: WeeChat 2.2]
16:53-!-marshmn [~matt@84.93.31.67] has quit [Ping timeout: 480 seconds]
17:02-!-anomie [~anomie@00018802.user.oftc.net] has quit [Remote host closed the connection]
17:10-!-bumbleVole [~bumblevol@pool-108-41-237-196.nycmny.fios.verizon.net] has joined #linode
17:10-!-bumbleVole is "bumbleVole" on #linode
17:11<nuevu>More Dallas routing issues in the last couple of minutes?
17:12<millisa>looks similar to me
17:12<staticsafe>aye, just got a bunch of alerts
17:12-!-gecco [~oftc-webi@23-117-194-49.lightspeed.tukrga.sbcglobal.net] has joined #linode
17:12-!-gecco is "OFTC WebIRC Client" on #linode
17:13<gecco>Hello linode, all of our VOIP phones just dropped from servers in DALLAS data center
17:13<millisa>gecco: we were just talking about how it looks similar to yesterday
17:16<gecco>yeah
17:17<gecco>this is exactly what happened yesterday
17:17<gecco>same time
17:17<gecco>hey Ryan, is anyone from netops here?
17:19<gecco>so I'm being told there is a huge ddos attack hitting dallas
17:19<nuevu>All of us running ping/mtr >.>
17:19<nuevu> /s
17:19<staticsafe>seems to be recovering, my alerts just cleared
17:19<nuevu>Likewise
17:22<nuevu>Spoke too soon, apparently.
17:22-!-logan [~logan@irc.protiumit.com] has quit [Read error: Connection reset by peer]
17:22-!-logan [~logan@irc.protiumit.com] has joined #linode
17:22-!-logan is "logan" on #linode #ceph-ansible #ceph
17:23<gecco>my phones have reregistered
17:23<gecco>linode is opening a status page on dallas
17:27<linbot>New news from status: Connectivity Issues - Dallas <https://status.linode.com/incidents/t0k0897r7ggw>
17:27<gecco>I'm out again
17:27<gecco>more ddos in dallas
17:28<@mcintosh>millisa: thanks for noting the plan column thing - will fix that
17:29<millisa>sure thing
17:32<gecco>we're still out.... phone end points going up and down
17:32-!-Roger_ [~oftc-webi@c-69-242-88-130.hsd1.de.comcast.net] has joined #linode
17:32-!-Roger_ is "OFTC WebIRC Client" on #linode
17:33-!-Roger_ [~oftc-webi@c-69-242-88-130.hsd1.de.comcast.net] has left #linode []
17:33<csnxs>patience, my friend
17:33<csnxs>im sure they already have people down at dallas throwing spanners at things until it works again
17:37<cache`>Seems to be recovering on our end
17:38<cache`>was there a post-mortem for yesterday?
17:38<gecco>we're recovered, looks like first hit was hard, second that just happened was about half as bad as the first
17:38<gecco>we're back to normal right now
17:38<cache`>thanks for the updates 👍
17:44<gecco>More DDOS activity
17:45<gecco>We're up, but they are hammering
17:50<gecco>We're going out again...
17:51<gecco>This latest attack is worse than the first two
17:58-!-redentor [~redentor@189.202.73.238.cable.dyn.cableonline.com.mx] has joined #linode
17:58-!-redentor is "realname" on #debian-mx #debian-es #debian #linode
17:59-!-Clayton [~oftc-webi@199.192.152.8] has joined #linode
17:59-!-Clayton is "OFTC WebIRC Client" on #linode
18:00<Clayton>Any idea when dallas will be 100%?
18:03<gecco>we just started to recover endpoints, we're about 100% back to normal now
18:08-!-ughugh [~oftc-webi@cpe-76-88-0-222.san.res.rr.com] has joined #linode
18:08-!-ughugh is "OFTC WebIRC Client" on #linode
18:08-!-dueyfinster [~dueyfinst@51-171-118-84-dynamic.agg2.rsl.rsl-rtd.eircom.net] has quit [Read error: Connection reset by peer]
18:09<ughugh>Any idea what the issues are at Dallas? 2nd day in a row.
18:12<warewolf>ohno
18:14<gecco>its a giant ddos attack
18:14<gecco>same shit as yesterday
18:15<gecco>took almost 250 of my phone end points down, I'd like to break someones face right now
18:15<ughugh>Christmas 2015 all over again. Well, not quite.
18:15<gecco>no, no, no
18:15<gecco>don't talk about that
18:15<gecco>I'm serious...
18:15<gecco>I lost... a few clients
18:15<gecco>really sad days
18:15<ughugh>You aren't the only one
18:15<gecco>Linode is not the same Linode today as it was then
18:16<gecco>they totally stepped up their game, private fiber backhaul to atlanta via Telex... major upgrades in infrastructure
18:16<gecco>those days are long gone I hope
18:16<Clayton>Yes I'm in the same boat with PBX
18:17<gecco>its a good home in dallas
18:17<gecco>first issues all year, first major issues since... 2015
18:17<gecco>clayton PM me, maybe we can exchange notes, I only operate in the atlanta area
18:17<Clayton>Im thinking about grabbing another linode and having it replicated and just change my DNS A record's when something like this happens
18:18<Clayton>another location*
18:18<gecco>yeah, I've not had enough issues where I want to do that yet, or run HA
18:18<gecco>simple has worked
18:18<Clayton>i agree
18:18<Clayton>i think it would be pretty simple that way
18:18<gecco>there is lylix, have you seen that
18:18<Clayton>i have not
18:18<gecco>expensive, but an option, they do a lot of voip
18:19<gecco>you won't get linodes network though
18:19<gecco>or the support
18:19<ughugh>Our site is too big to just switch DNS. But we do have fail-overs in place
18:19<gecco>ughugh you doing voip too?
18:19<ughugh>Nope
18:19<gecco>sites?
18:19<ughugh>Just a fairly decent sized site
18:19<gecco>so why not aws?
18:20<ughugh>Some of our stuff does
18:20<ughugh>But we have also found issues with speed between their EC2 and RDS issues as well as enormous bandwidth costs.
18:20<ughugh>EC2 and RDS instances
18:20<gecco>yeah
18:21<gecco>they redefine "charge for everything"
18:21<Clayton>AWS is terrible
18:21<Clayton>linode smokes them on performance and price
18:21<gecco>big time
18:21<gecco><3 linode
18:21<Clayton>so fellas i think i will do that
18:21<ughugh>It's also a maintenance nightmare getting everything setup. A million different services with a million different options all in a nice non-intuitive dashboard
18:21<gecco>I'll sell these guys all day long
18:21<Clayton>Create a linode in Cali and have it sync with dallas
18:21<ughugh>We had AWS issues lastr week
18:21<Clayton>when stupid issues come up like this change dns record and be up in 10 min
18:22<Clayton>i have 157 end points on this server
18:22<ughugh>I can pretty much switch in 5 minutes assuming everything is in place.
18:22<Clayton>with dns?
18:22<ughugh>Yup
18:22<ughugh>Cloudflare is wicked fast in that regard
18:22<gecco>they are
18:22<Clayton>i am using godaddy for domain
18:22<gecco>clayton are you using SBC's?
18:22<Clayton>sounds like i need to switch
18:23<Clayton>sorry what are sbc's
18:23<gecco>godaddy is fast sometimes, but their NS and DNS servers go down once or twice a year
18:23<gecco>session border controllers, we use edgewater edgemarc's
18:23<gecco>that's what does our QOS
18:25<ughugh>We use GoDaddy for our China domain names, that's it.
18:29<Clayton>was hoping the issue was dallas was resolved
18:29<Clayton>still shitty here
18:30-!-dueyfinster [~dueyfinst@51-171-118-84-dynamic.agg2.rsl.rsl-rtd.eircom.net] has joined #linode
18:30-!-dueyfinster is "NG" on #linode
18:33<csnxs>who told you its a ddos attack? haven't seen it actually mentioned by linode
18:33<ughugh>Linode is very private about what is happening, although after Christmas 2015 they promised to be more transparent. They haven't been which is one of my big disappointments.
18:36<ughugh>Heh, been trying to find real time DDoS attack maps. They are pretty, but they all show different things.
18:37<nate>So basically you don't actually have any evidence it's a DDoS and prefer to just say that linode is "being private" even though they've been pretty clear when it's an attack issue
18:37<nate>lol
18:38<ughugh>I'm not the one who said it was a DDoS attack
18:39<ughugh>And I don't see anything in their status updates which indicates what is going on. So, that's pretty private. Or they just don't know.
18:39<nate>Ah, but you seem to be backing that it probably is? Even though there seems to be no offical statement about it. As far as the 2015 holiday attacks go, I remember them going into pretty detailed blog post about the attacks after it all
18:40<ughugh>Nope, I didn't back anything. Please don't assume things.
18:40<gecco>They were pretty straight with me, I was all the way to the top of network engineering, they even assigned someone from the ATL data center to look into the issue
18:40<nate>"Linode is very private about what is happening, although after Christmas 2015 they promised to be more transparent. They haven't been which is one of my big disappointments." <-- leaves pretty minimal room for assumption, perhaps instead you should word things better?
18:41-!-dueyfinster [~dueyfinst@51-171-118-84-dynamic.agg2.rsl.rsl-rtd.eircom.net] has quit [Quit: Goodbye!]
18:41<ughugh>Perhaps.
18:41<gecco>turned out, f'ing GNAX which sold out to Zayo Group enabled some kind of big ddos system on their data center, we had to flee atlanta to dallas to escape it, kept dropping our phone packets so phones would just randomly drop off
18:41<gecco>for as little as the bill was (and still is) they were pretty good
18:41<gecco>I just wish we could pay more money for even more attention when problems happen
18:42<nate>I would imagine professional or managed accounts may get slightly extra priority
18:42<gecco>well, what good does it do you in a ddos ?
18:43-!-dueyfinster [~dueyfinst@51-171-118-84-dynamic.agg2.rsl.rsl-rtd.eircom.net] has joined #linode
18:43-!-dueyfinster is "NG" on #linode
18:44-!-dueyfinster [~dueyfinst@51-171-118-84-dynamic.agg2.rsl.rsl-rtd.eircom.net] has quit []
18:44<gecco>we;re back up to 100%, have been for the last 20
18:44<gecco>looks like the shows over
18:44<gecco>hopefully, see you guys in a few years?
18:44<Clayton>mines still a little sketchy
18:44-!-dueyfinster [~dueyfinst@51-171-118-84-dynamic.agg2.rsl.rsl-rtd.eircom.net] has joined #linode
18:44-!-dueyfinster is "NG" on #linode
18:52<ughugh>A few years sounds good to me :)
18:52<ughugh>Maybe I'll have moved on by then.
18:52<ughugh>I doubt it
18:53-!-dueyfinster [~dueyfinst@51-171-118-84-dynamic.agg2.rsl.rsl-rtd.eircom.net] has quit [Quit: Goodbye!]
18:53<Clayton>working good right now
18:54-!-dueyfinster [~dueyfinst@51-171-118-84-dynamic.agg2.rsl.rsl-rtd.eircom.net] has joined #linode
18:54-!-dueyfinster is "NG" on #linode
18:58-!-dueyfinster [~dueyfinst@51-171-118-84-dynamic.agg2.rsl.rsl-rtd.eircom.net] has quit []
19:00<ughugh>duefinster's ISP is having problems though :)
19:00-!-dueyfinster [~dueyfinst@51-171-118-84-dynamic.agg2.rsl.rsl-rtd.eircom.net] has joined #linode
19:00-!-dueyfinster is "NG" on #linode
19:01-!-dueyfinster [~dueyfinst@51-171-118-84-dynamic.agg2.rsl.rsl-rtd.eircom.net] has quit [Remote host closed the connection]
19:02-!-dueyfinster [~dueyfinst@51-171-118-84-dynamic.agg2.rsl.rsl-rtd.eircom.net] has joined #linode
19:02-!-dueyfinster is "NG" on #linode
19:06-!-dueyfinster [~dueyfinst@51-171-118-84-dynamic.agg2.rsl.rsl-rtd.eircom.net] has quit [Remote host closed the connection]
19:07<linbot>New news from community: How can I configure a Linode as a proxy? <https://www.linode.com/community/questions/17720> || How can I configure a Linode as a proxy? <https://www.linode.com/community/questions/17719>
19:08-!-ntox [~textual@164.51.190.98] has quit [Ping timeout: 480 seconds]
19:12<ughugh>Monitoring
19:19-!-dueyfinster [~dueyfinst@51-171-118-84-dynamic.agg2.rsl.rsl-rtd.eircom.net] has joined #linode
19:19-!-dueyfinster is "NG" on #linode
19:24-!-gecco [~oftc-webi@23-117-194-49.lightspeed.tukrga.sbcglobal.net] has quit [Quit: Page closed]
19:31<Clayton>my pbx is jackin up again
19:32<cache`>^ having issues again here
19:32<Clayton>same
19:33*ughugh sighs
19:38-!-dueyfinster [~dueyfinst@51-171-118-84-dynamic.agg2.rsl.rsl-rtd.eircom.net] has quit [Remote host closed the connection]
19:40-!-Maz [~oftc-webi@103.97.220.8] has joined #linode
19:40-!-Maz is "OFTC WebIRC Client" on #linode
19:40-!-Maz [~oftc-webi@103.97.220.8] has quit []
19:51-!-rainbow [~ssmith@00020809.user.oftc.net] has quit [Quit: Sorry, ZNC derped!]
19:53-!-fstd [~fstd@xdsl-87-78-203-4.nc.de] has joined #linode
19:53-!-fstd is "fstd" on #oftc #linode #kernelnewbies
19:54-!-rainbow [~ssmith@00020809.user.oftc.net] has joined #linode
19:54-!-rainbow is "Samantha "Rainbow" Smith" on @#linode-tavern #linode
19:54<Clayton>does atlanta location work ok for pbx?
19:56-!-rainbow [~ssmith@00020809.user.oftc.net] has quit []
20:00-!-fstd_ [~fstd@xdsl-78-35-91-83.nc.de] has quit [Ping timeout: 480 seconds]
20:01-!-rainbow [~ssmith@00020809.user.oftc.net] has joined #linode
20:01-!-rainbow is "Samantha "Rainbow" Smith" on @#linode-tavern #linode
20:13<Clayton>Hows everyone working on dallas?
20:14<t27duck>Don't think anyone is. Status page reports the issue resolved.
20:14<t27duck>been fine for me
20:14<ughugh>Seems OK at moment.
20:14<ughugh>Wonder what was going on
20:37<Clayton>would like to know if it could potentially be an intermittent issue.. i will switch to another location
20:38<Clayton>i went ahead and duplicated my server to another linode in atlanta so i can make the switch in a hurry if i need ot
20:38<Clayton>to*
20:43-!-Clayton [~oftc-webi@199.192.152.8] has quit [Remote host closed the connection]
20:44-!-Clayton [~oftc-webi@199.192.152.8] has joined #linode
20:44-!-Clayton is "OFTC WebIRC Client" on #linode
20:45-!-Clayton [~oftc-webi@199.192.152.8] has quit []
20:47-!-bumbleVole [~bumblevol@pool-108-41-237-196.nycmny.fios.verizon.net] has quit [Remote host closed the connection]
20:47-!-bumbleVole [~bumblevol@pool-108-41-237-196.nycmny.fios.verizon.net] has joined #linode
20:47-!-bumbleVole is "bumbleVole" on #linode
20:48-!-ughugh [~oftc-webi@cpe-76-88-0-222.san.res.rr.com] has quit [Quit: Page closed]
20:58-!-bumbleVole [~bumblevol@pool-108-41-237-196.nycmny.fios.verizon.net] has quit [Remote host closed the connection]
20:58-!-anomie [~anomie@00018802.user.oftc.net] has joined #linode
20:58-!-anomie is "Anomie" on #linode
20:58-!-bumbleVole [~bumblevol@pool-108-41-237-196.nycmny.fios.verizon.net] has joined #linode
20:58-!-bumbleVole is "bumbleVole" on #linode
21:02-!-waltman [nunya866@c-68-81-107-5.hsd1.pa.comcast.net] has quit [Quit: leaving]
21:07-!-anomie [~anomie@00018802.user.oftc.net] has quit [Remote host closed the connection]
21:12-!-waltman [nunya224@c-68-81-107-5.hsd1.pa.comcast.net] has joined #linode
21:12-!-waltman is "Walt Mankowski" on #linode
21:24-!-thiras [~thiras@195.174.215.70] has quit [Ping timeout: 480 seconds]
21:54-!-Clayton [~oftc-webi@199.192.152.8] has joined #linode
21:54-!-Clayton is "OFTC WebIRC Client" on #linode
21:54<Clayton>Everyone doing ok on dallas linode?
22:03-!-Clayton [~oftc-webi@199.192.152.8] has quit [Remote host closed the connection]
22:44-!-redentor [~redentor@189.202.73.238.cable.dyn.cableonline.com.mx] has quit [Remote host closed the connection]
23:26-!-schwa [~laptopdud@pool-108-24-114-94.cmdnnj.fios.verizon.net] has joined #linode
23:26-!-schwa is "purple" on #linode
23:34-!-schwa [~laptopdud@pool-108-24-114-94.cmdnnj.fios.verizon.net] has quit [Ping timeout: 480 seconds]
23:36-!-schwa [~laptopdud@pool-108-24-114-94.cmdnnj.fios.verizon.net] has joined #linode
23:36-!-schwa is "purple" on #linode
23:45-!-schwa [~laptopdud@pool-108-24-114-94.cmdnnj.fios.verizon.net] has quit [Ping timeout: 480 seconds]
23:54-!-bumbleVole [~bumblevol@pool-108-41-237-196.nycmny.fios.verizon.net] has quit [Remote host closed the connection]
23:55-!-bumbleVole [~bumblevol@pool-108-41-237-196.nycmny.fios.verizon.net] has joined #linode
23:55-!-bumbleVole is "bumbleVole" on #linode
23:58-!-bumbleVo_ [~bumblevol@pool-108-41-237-196.nycmny.fios.verizon.net] has joined #linode
23:58-!-bumbleVole [~bumblevol@pool-108-41-237-196.nycmny.fios.verizon.net] has quit [Read error: Connection reset by peer]
23:58-!-bumbleVo_ is "bumbleVole" on #linode
---Logclosed Wed Feb 06 00:00:52 2019