Scalyr Blog

Scalyr is great, but I'm not sure if it's any cheaper. https://www.scalyr.com/

Scalyr supports live tail (in both classic and new UI modes): http://www.scalyr.com (*disclaimer: I'm one of the co-founders)

Scalyr (https://www.scalyr.com) does this, you should give it a try. You can monitor and analyze multiple servers in one place on one dashboard. (Full disclosure: I work for Scalyr.) If you have any questions or feedback I'd love to hear it!

SEEKING FREELANCER - San Francisco Bay Area, REMOTE is acceptable Windows Python programmer with focus on creating Windows Services https://www.scalyr.com We are an early stage startup building a log management, monitoring, and alerting service based on our experience running cloud services at Google. We are seeking a freelancer to adapt the agent program that our customers run on their servers to collect system metrics and server logs. It is written in Python and currently runs on Linux so we are looking to adapt it for Windows. This is a great opportunity to work with a small, very engineering-focused company at an early stage of development working with very interesting technologies. The project will have a big impact on our customers. Plus, the agent is open sourced so this work benefits the community and will give the developer something they can point at for future employers. Finally, this project could result in more ongoing work in the months / years to come. The main tasks will include: * Create a build process that will bundle the Python agent as a Windows executable and run it as a Windows service (using something like py2exe) * Create an installer * Develop a mechanism to allow our users to easily update the agent when new releases are made * Write code to collect relevant system metrics from the Windows server We are looking for candidates who meet the following requirements: * Windows expert, focusing on Window Services and best practices *…

Please consider Scalyr: disclosure I work there, but objectively I will be surprised if someone beats us on price, focused log analytics - try and judge for yourself. https://www.scalyr.com

Scalyr | DevOps, Backend Engineers | San Mateo, CA | ONSITE I've built half a dozen startups, including Writely (aka Google Docs), and I can honestly say that Scalyr is my favorite by far. We are the blazing fast log management platform built for the DevOps front line. We built our technology stack from the ground up, optimizing it for speed and scalability. With over 180 customers, real revenue and only 32 employees, the time to join is now! DevOps Engineer: Join the core engineering team to help us scale our growing infrastructure, ingesting tens of TBs of logs every day. Backend Engineer: We've built a NoSQL data engine from scratch that searches test at 1TB/second. How'd you like to help us scale and optimize to multiple TB/second? Sound interesting? apply now: https://www.scalyr.com/jobs

I'm using http://logentries.com The one I really wanted to use/like was http://scalyr.com. However even after their redesign, I still can't use their query language. With LogEntries, it's pretty natural.

Also, see the follow-up post: http://blog.scalyr.com/2014/07/impossible-problems-often-arent/ In the end, they did have to do some pre-computes (which is better than indexing, but still not 100% on-the-fly resolution).

Scalyr | San Mateo | Site Reliability Engineer | Full-Time, Onsite We are looking for a Site Reliability Engineer who can help keep our uptime promise to our customers by making sure we meet our SLOs and can help our engineering teams ship software to our customers fast and with quality. On this job, you will have an amazing opportunity to drive outcomes that improve reliability, stability and cost efficiency of Scalyr. We are looking to add a SRE with prior extensive operations experience for a SaaS product who can drive deployment re-architecture with focus on self-service and automation. Someone who has driven continuous deployment, has run incident post-mortems, has provided feedback to engineering architecture decisions and has automated repetitive operational tasks would be a great fit. You will join a like minded team of awesome SRE engineers who help run our operations smoothly at scale. We value good written communication skills, data driven decisions and a keen eye for continuous improvements. You’ll help simplify, have a passion for new ideas and know how to execute iteratively towards the final goal. We value candor and collaboration. SRE and DevOps are two titles loosely used in the industry. A SRE engineer at Scalyr defines and provisions the common set of tools the engineering teams to use and facilitate dev <-> ops collaboration by consulting and driving best practices. SRE at Scalyr is also responsible for uptime and providing feedback to engineering on…

We use and like https://www.scalyr.com/.

Brute force search is how Scaylr does it. Here’s an article they wrote about their implementation. https://www.scalyr.com/blog/searching-1tb-sec-systems-engineering-befo…

Not too sure about Angular / JS, but instead of console.log() I'd use something that you can determine a log level for i.e. console.error / console.info / console.debug. Then you can determine what to ship out to a logging tool like logstash/logdna based on the environment. Edit: Something like this https://www.scalyr.com/blog/getting-started-angular-logging/

I keep this on my bookmarks: [1] (that I fished from this other blog post [2]). Not sure if any actual search tools other than Scalyr use that specific implementation, but it makes for an interesting read anyway if you are into substring search algorithms (the algo described improves on Boyer-Moore). 1: http://volnitsky.com/project/str_search/ 2: https://www.scalyr.com/blog/searching-1tb-sec-systems-engineering-befo…

Something went wrong with the third link, here it is for anyone looking: https://www.scalyr.com/company/jobs/sales-engineer

The article [1] was about a log search/parsing service, so 20 GB of log files/second. Log files are usually text. [1] http://blog.scalyr.com/2014/05/searching-20-gbsec-systems-engineering-…

The quick bitcoin mining spin-ups, and the initial email from Amazon about the key going public, are (probably) thanks to the github events firehose. https://github.com/hootware/github-firehose-node http://blog.scalyr.com/2013/10/exploring-the-github-firehose/

https://www.scalyr.com/dash?page=Github-Statistics is broken. "Operation not permitted (Read Configuration permission required)"

Direct link to the calculator -- https://www.scalyr.com/cloud/

That's good advice. As the saying goes: "trust, but verify". Regarding your performance issue - most platforms (including dotCloud) enforce ram and cpu separation between nodes, but are vulnerable to IO contention at some level. This is also true for EC2 if you use EBS: your standalone instances will almost certainly, at some point, suffer from degraded and inconsistent performance because another instance is competing for IOPS [1]. You can avoid this with the new "provisioned IOPS" volumes [2], or by skipping EBS altogether for stateless components. [1] http://blog.scalyr.com/2012/10/16/a-systematic-look-at-ec2-io/ [2] http://aws.amazon.com/about-aws/whats-new/2012/07/31/announcing-provis…

The basecamp page (http://basecamphq.com/uptime) tells me at a glance what sort of service I'm getting. The scalyr page (https://www.scalyr.com/monitor) doesn't (even after three glances). The sort of data presented there may be useful in diagnosing problems, but the presentation is ill suited to tell me if "service is bad" at a glance. I'd favor a service that offers both, with a high-level "basecamp" overview that offers the option to get the "scalyr" details when I need to.

Scalyr | San Francisco/San Mateo | REMOTE | Multiple Roles - UI/Software Engineer/SRE | Full Time | https://www.scalyr.com/careers/ Scalyr is a log management SaaS, a part of monitoring & observability stack. Our vision is "to be the leading source of community engagement on the use and impact of observability", and we have a compelling log management product that we are famous for! [store and search petabytes of data, blazing fast queries (sub 200ms) and high ingestion rates (TB/s)]. Scaly's technology stack is built from the ground up, optimizing it for speed and scalability. We are growing in terms of both data we host and customers! If you are a frontend engineer (React, GraphQL) or a generalist software engineer who wants to be part of a hyper growth startup and want to work with talented engineers who challenge you, please reach out to us. We are upgrading our infrastructure and seek the help of a DevOps engineer and thinker to transform our operations to a modern tech stack, learn from it and improve the product for our customers. We run on AWS. We value transparency and collaboration. If you are interested, please get in touch with Lakshmi - saasy_log=scalyr mail -s "Hire me" jobs@${saasy_log}.com Office is in San Mateo close to caltrain station. We can consider remote post COVID too! We offer 100% employer paid medical benefits among other things! We value you and your family! PS: If you use silver searcher or ripgrep or ack to grep through your code,…

> Retries are tricky because depending on how the job is implemented they can cause more harm than good. Ah, yes, there's nothing like bringing capacity back online only to have it crushed by all your customers retrying at the same time. AWS got bit hard by this[3] but there's a blog post[1] about it, which is linked to by the docs for their client software[2]. [1] https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitt… [2] https://docs.aws.amazon.com/general/latest/gr/api-retries.html [3] https://aws.amazon.com/message/5467D2/ ... basically DynamoDB is a fundamental service for AWS and had implemented some new streams features. This all appeared to be working, but they were running closer to capacity than intended, and when a cluster went out this caused a cascading failure. And see this which linked to that RCA: https://blog.scalyr.com/2015/09/irreversible-failures-lessons-from-the…