Incapsula Blog

To summarize the discussion here so far: 1- For small attacks you can optimize your stack, cache your content and use a provider that allows you to quickly scale and add more servers to handle the traffic. Do not use Linode or Digital Ocean as they will null route you. OVH, AWS and Google are the ones to go with. 2- Use a DDoS mitigation / CDN provider that will filter the attacks and only send clean traffic back to you. The ones recommended so far: https://cloudflare.com https://sucuri.net https://incapsula.com

This is both fantastic discussion of the ability to quickly check for vulnerabilities in ways never possible at scale before and also discussion of a little-known vulnerability that even the security experts were not aware of. The security guys at Imperva Incapsula just wrote up how they protect their system against this here (https://www.incapsula.com/blog/http-host-header-fix.html) – and in their tests, the only vulnerability they found was their own tests. But they wouldn’t have done it until the BlackHat presentation.

This has been offered by their competitor Incapsula for some time under the name 'Global Server Load Balancing' https://www.incapsula.com/load-balancing-failover.html

I'm late, but for this there are companies that sit in the middle and block crawlers, spambots and known-malicious addresses. Here's one I use: http://www.incapsula.com/ So far I've been happy with it!

This is almost certainly a prank or a parody. DNS for ddossite.com points to IP 199.83.134.131. "whois 199.83.134.131" shows the IP is inside an IP block owned by "Incapsula Inc". Googling the company name yields http://www.incapsula.com/, which says they are a network security company and they provide DDoS protection.

"google can disregard nofollow on a case by case basis" This is somewhat correct. While Google won't disregard noindex/nofollow directives (or robots.txt Disallow) it can - and will - show "blocked" pages in SERP if the quarry is specific enough or a the page is "strong" (lots of inbound links/strong social signals and etc...) However, it will not show in page info (hide description and title) and use alternatives sources of information to fill those out instead. For example: it can use DMoZ info instead of the real one. Source: http://www.incapsula.com/the-incapsula-blog/item/395-new-snippet-puts-…

I’ve been working on a bot monitoring platform - hoping to launch soon. Problem: Up to 61.5 percent[1] of website visitors are bots, of which 30.5% are malicious. The presence of bots lead to content theft, malware injection, server hijacking, spam links and DDoS attacks. Website owners are often unaware of their existence and the extent of their effects. Solution: A platform to monitor and analyse bot traffic. We alert you to attacks, pinpoint affected pages and identify and analyse all bots. To retrieve traffic data we use pixel tracking & JavaScript tracking code. To detect bots we look for abnormalities in a series of traffic data such as HTTP header fingerprinting, JavaScript footprint, timestamps, IP reputation etc. With the results from our platform you can better implement methods of prevention such as blocking IP addresses/user agents or go about including CAPTCHAS or similar where necessary. If this is something you would be interested in I’d love to hear what you think :). [1] http://www.incapsula.com/blog/bot-traffic-report-2013.html

Hi, I work for Incapsula. Yesterday we've addressed this study in our blog. http://www.incapsula.com/the-incapsula-blog/item/699-incapsula-pentest… To make a long story short, this detailed report helped us introduce several patches to our WAF - but we feel that some of the issues were mostly theoretical and patching them can lead to security eroding false positives. ModSecurity is a great security solutionand we are happy to be compared to it. We feel that we perhaps offer a more complete package (bot filtering, acceleration, user friendly GUI and easy setup, support, etc). Still, you can't beat Free and if you are looking for OS security solution, ModSecurity is defiantly a name you should consider. On a personal note, I wish that more vendors would take security a bit more seriously - especially if they claim to provide it to thousands of clients. Web evolution is motivated by consumer trust and if we loose this trust we will also loose the cash flow that keeps Internet evolving. Providing faulty security will do just that.

Why are you looking to scrape? Here's a list of some scraper bots: https://www.incapsula.com/blog/web-scraping-bots.html What about Botscraper: http://www.botscraper.com/

Sure, go join the realms of shady SEOs and malware, if I want to really stop you I'll know you're not coming from a Google IP range. https://www.incapsula.com/blog/was-that-really-a-google-bot-crawling-m… However, consider what your ultimate end game is, if it's a website you expect visitors to find through Google or the Play store, good luck once web masters start reporting your misbehaving "Googlebot" crawler.

Ahrefs crawls actually more than any search engine other than Google: https://www.incapsula.com/blog/most-active-good-bots.html And Bing, Yandex, and Yahoo have active crawlers.

It turned out to be due to Incapsula web accelerator/protector [0]. > Incapsula is a service that secures and accelerates the website you were trying to reach. It works fine in Firefox/Chrome, I was trying to open it using qutebrowser so maybe it's a problem with QtWebKit? Anyways, my fault for jumping to conclusions (thinking the site was hugged to death). [0] https://www.incapsula.com/why-am-i-seeing-this-page.html

So many devices are now connected to the Internet and potentially vulnerable. The basics definitely matter – changing default passwords, ensuring our internet is hosted on DDoS protection servers, etc. But who was going to warn the traffic engineer that their security camera is vulnerable or the new parents whose IP-connected baby monitor gets scanned by foreign hackers. We just want things to work and don’t realize that we’re at risk – even if our device is the target and not ourselves. Have you checked out this Mirai vulnerability scanner? Something everyone should do – whether a random home user or a large enterprise (and how many have CISOs?). It scans your IP and can pinpoint vulnerable devices: https://www.incapsula.com/mirai-scanner.html

> Per the article Not quite, it says "If the attacker has a bigger fire hose of data than the defender has, the attacker wins" and "the size and scale of these probes—and especially their persistence—points to state actors" which is not quite the same as saying you need to own the bandwidth. For example, DNS amplification can be used "to turn initially small queries into much larger payloads, which are used to bring down the victim’s servers". https://www.incapsula.com/ddos/attack-glossary/dns-amplification.html So maybe there are other techniques which might allow for similar leverage. Neither is the article conclusive about "state actors", they are merely "pointed to". As for expertise ... I don't doubt there are people out there who have it or might acquire it. So it's still an interesting question imo.

Here's the most prominent one that's happening now: http://www.incapsula.com/blog/semalt-botnet-spam.html Basically referrer spam.

Any chance it's related to this? http://www.incapsula.com/blog/world-largest-site-xss-ddos-zombies.html Security company found a vulnerable Alexa Top 50 site where someone was able to inject XSS code in the comments section creating "DDoS Zombies" of the visitors.

...of all website traffic... The report [1] does not include any non-http traffic, which excludes all heavy content like torrent, Netflix, music streaming, etc. [1] http://www.incapsula.com/the-incapsula-blog/item/820-bot-traffic-repor…

I wondered how they distinguished bots. The only info I found on the study [1], on a quick search, says: "For the purpose of this report we observed 1.45 Billion bot visits, which occurred over a 90 day period. The data was collected from a group of 20,000 sites on Incapsula’s network" So, in the first place, not necessarily representative of the web generally. And it does not say how they detect bots. Of course some declare themselves [2]; otherwise I guess one would have to rely on captchas [3], which are known to be of limited reliability and subject to an arms race. 1. http://www.incapsula.com/the-incapsula-blog/item/820-bot-traffic-repor… 2. kristopolous on this page, '(bot|spider|crawl|aggregator)' 3. which Incapsula does, as a condition of access to its summary of its previous similar research.

Wrote a blog about this just a week ago - here you go : http://www.incapsula.com/the-incapsula-blog/item/303-seo-cdns-and-clou…

Here's a thought experiment. This one requires some long-term thinking, outside the box and well past recent history and the status quo. What if the majority internet usage is non-interactive, from so-called "bots", what we may refer to as "automated use". Google and Facebook, among others, rely on the use of automation and "bots". The non-interactive clients ("bots") being used by these companies are not asked to solve captchas. (In turn, after collecting data from public sources, these websites attempt to prohibit the use of automation by their users wishing to access it. What is interesting is that neither company provides any definition of "automated" nor any clearly stated limits on the speed at which a user may access resources or the quantity of resources they may access in a stated time period. One might be apt to find such limits associated with an "API".) In 2013 an Incapsula report suggested that the majority of internet usage is in fact automated and not "malicious"^1 -- what if public information sources on the internet catered to the use of automation rather than trying to limit such use, e.g., with speed bumps^2 like "captchas". What if servers treated all clients equally, instead of having data forcibly collected by a few large clients that receive preferential treatment, then siloed and protected from "automation". What effects would this have on "centralisation" and levelling the playing field. "Do not ask for permission, ask for forgiveness." …