During my 20+ years in network defense—starting as an Air Force systems operator and evolving through contractor work in information warfare and offensive/defensive security—I've seen this pattern repeat so consistently it's become predictable. An analyst reaches out, frustrated and drowning, asking a question that sounds operational but is really existential: "Is this normal?"
Last week, I was surfing Reddit and I noticed a post in r/cybersecurity which captured this crisis perfectly. An L1 SOC analyst working at an MSSP posted a plea that should alarm every security leader: "L1 SOC analyst here—drowning in false positives."
The details are devastating: tier-1 analysts dealing with thousands of alerts per day, 90%+ false positives, no structured approach to tuning, everyone just "experimenting" with thresholds and whitelists. The analyst's conclusion? "I feel like I'm stuck in an endless loop of closing the same false positives every day and as a result, real alerts often get missed."
But what happened in the comments—the community's response—reveals something far more important than one struggling analyst. It exposes the systemic failures in how we structure, scale, and operate modern SOCs. Let me unpack what this conversation really tells us.
The Surface Problem: False Positives
The original poster laid out their situation with brutal honesty:
> "I'm working as an L1 SOC analyst at an MSSP, where we handle multiple clients. The main issue I'm running into is the insane volume of alerts, thousands of offenses per day, and honestly, 90%+ are false positives... There is no structured approach for rule creation or fine-tuning. Everyone just experiments. some people tweak thresholds, others disable rules, some whitelist entire domains or IP ranges (ofc after receiving approval from the customer). It feels like chaos with no methodology behind it."
This isn't just one analyst's experience. One commenter confirmed: "I literally supported government and large, well-known private organizations locally. The amount of alerts that would come through was 98% false positives."
On its face, this looks like a technical problem—bad detection logic, poorly tuned SIEM rules, inadequate alert filtering. The technical crowd jumped on that angle immediately. But the responses revealed something far more profound.
What the Comments Actually Revealed: Five Systemic Failures in the SOC
1. The Process Vacuum
One of the commenters asked the most important question first: "What happens when you tell your manager? They should help with process and structure. How large is your MSSP headcount wise?"
This cuts right to it. The analyst isn't asking for technical help—they're asking if "chaos is normal". And the answer from experienced practitioners was unanimous: No, this is a massive process failure.
Another commenter put it plainly: "Sounds like the SOC need a sit down meeting to get on same page. Tune the alerts. Develop consistent procedure. Some automated responses."
Another commenter with MSSP experience spelled out what mature operations look like: "No SOC should have thousands of alerts a day. In my fully mature MSSP SOC, we get around 5k per month. We achieve that through heavy tuning. You have to have a detection engineering team and a clear process for going about tuning detections... Every day we push out at least 10 tuning requests for noisy rules."
Mature SOCs have:
- Detection engineering teams (not just analysts tweaking rules ad-hoc)
- Clear tuning processes with request workflows and review procedures
- Proactive volume management ("Seniors and leads need to be looking for high alert rules and finding ways to tune out useless crap")
- Formal approval chains ("There should only be a select few people who can implement tuning requests")
Immature SOCs have what this analyst described: chaos with no methodology.
2. The Knowledge Capture Crisis
One comment on this thread stood out to me because it connects directly to something I learned early in my career as a network administrator:
> "It also helps to be interested in the habits and personalities of different software companies. It is an old system administrator habit that becomes useful here. I used to skim the full report on quiet days so I would recognize the normal behaviors of ordinary programs."
This resonated immediately with me. A few years ago, I was reviewing output from a new machine learning model we were testing for anomaly detection. Within minutes of looking at the flagged events, I started calling out patterns: "This is standard Windows Update behavior. This is Symantec doing its thing. This cluster is just backup jobs running."
The engineers building the model looked at me, puzzled: "How do you know that so fast? You barely looked at the data."
The answer went back to those early days as a network administrator. You learn to recognize the rhythms. Microsoft patch management has a distinct signature—specific processes, predictable timing, characteristic network patterns. Cisco automated updates look completely different. Backup software generates PowerShell executions that follow recognizable templates. Even malware families develop "personalities" if you see enough of them.
You don't learn this from documentation or training courses. You learn it by watching systems operate during the quiet hours. By digging into anomalies and tracing them back to their source. By experimenting with tools in lab environments to understand their normal behavior before you see them flagged as suspicious in production.
This is exactly the kind of expertise that separates effective analysts from overwhelmed ones. It's pattern recognition built over hundreds or thousands of hours of observation. It's understanding that legitimate administrative tools have predictable behaviors, and deviations from those patterns mean something worth investigating.
But here's the problem:
This knowledge lives only in the heads of experienced analysts. And overwhelmed analysts won't have the time to make these kinds of observations.
When I was coming up through network administration and transitioning into security operations, I built this same mental library. You learn the rhythms. You recognize patterns. You develop an intuition for what's normal versus what's anomalous. It takes months or years of pattern recognition.
The L1 analyst posting this plea doesn't have that knowledge base yet. And their organization has no mechanism to transfer it. They're expected to somehow osmose this expertise through... what, exactly? Escalating alerts and hoping senior analysts explain their reasoning? (Spoiler: they don't have time either.)
One commenter nailed the fundamental challenge: "That's just how it works. It takes serious data science skills and time to develop something that translates individual signals and events into something more meaningful for an analyst. If your system leaves that to you, there is no winning with it."
3. The Tool Maturity Gap
Multiple commenters pointed to the underlying technology problem, and it's not what you might think. The issue isn't bad tools—it's immature tool implementations.
One commenter with Splunk experience explained: "If you go to alert level there will be lots, they are NOT false positives and so aren't yours, they are just signals that only mean something in a context. But that is ES [Enterprise Security], if you have just vanilla Splunk it is not a SIEM at all, it needs about 2-4 man-years of development in my estimates to get to ES level, without UBA."
Another added bluntly: "Sounds like you're using rubbish old tech... If your team didn't spend man years setting it up, you get 10s of thousands of rubbish alerts per day."
This is the dirty secret of security operations: buying a SIEM or EDR platform is maybe 20% of the solution. The other 80% is:
- Baseline tuning for your specific environment
- Detection logic customization
- Integration of contextual data sources
- Alert enrichment and correlation
- Continuous refinement based on operational feedback
Organizations treat security tools like they're plug-and-play appliances. They're not. They're platforms that require significant engineering investment to become operationally effective.
But here's what happens in practice: The SIEM gets deployed, generates thousands of alerts, and management says "Great! Now hire L1 analysts to handle all those alerts." The L1 analysts—who don't have the engineering expertise or organizational authority to fix the root problem—are left drowning.
4. The Tier Structure Breakdown
Several comments touched on what should be an obvious question: Where are your T2 and T3 analysts?
One commenter pointed out: "L1 implies there are L2-Ln? Ask up the chain. Do not ask another L1. If there is no answer from higher up you can either try to get an SOP implemented or you can quit."
This exposes a fundamental misunderstanding of how tiered SOC structures are supposed to work. The tiers aren't just about job titles or pay grades—they're supposed to represent expertise levels and knowledge transfer mechanisms.
In a healthy SOC:
- *L1 analysts handle high-volume, structured investigations with clear playbooks
- L2 analysts tackle more complex investigations and build those playbooks
- L3 analysts and detection engineers design detection logic, tune rules, and capture investigative expertise in automated workflows
The knowledge should flow down: L3 creates sophisticated detection logic → L2 develops investigation procedures → L1 executes structured investigations at scale.
But in this analyst's SOC, none of that is happening. Everyone at every tier is just "experimenting." The knowledge isn't being captured, structured, or transferred. The tiers exist in name only.
5. The Customer Relationship Problem
This one caught me off guard until I thought about it more carefully. Multiple commenters mentioned the customer dynamic:
"Each customer is gonna be special and make a bunch of trash. You have to work with them at first to tune the alerts, whitelisting the normal processes that are throwing false positives."
"At least in the experience from the customer side of a SOC experience, we have meetings with the vendor to discuss changes and when initially setting up there was a learning and tuning period to tweak things."
Here's what's happening: In MSSP environments, each customer has unique infrastructure, unique business processes, and unique "normal" behavior. A monitoring rule that works perfectly for Customer A creates a false positive storm for Customer B.
But this analyst's organization appears to have no structured process for customer-specific tuning. No formal learning period. No documented baseline development. Just ad-hoc whitelisting and hoping it doesn't create security gaps.
One commenter expressed the customer concern perfectly: "Feel free to correct me if there is some kind of auditing or approval process but I'd be pretty peeved if my company missed an account compromise or other threat alert because a single technician at our SOC vendor thought it was a good idea to do a broad whitelist."
This is the nightmare scenario: junior analysts making judgment calls about what to suppress, without senior review, without understanding the security implications, driven purely by the need to reduce alert volume.
Mapping These Patterns to the Broader Industry Crisis
Let me connect these Reddit observations to the broader patterns I've seen across the security operations landscape.
The False Positive Problem is a Symptom, Not the Disease
Organizations treat false positives as a technical problem: "We need better detection logic." But the Reddit thread reveals that false positives are actually a knowledge problem and a process problem.
Better detection logic helps, but it's not sufficient. Why? Because:
1. Environments are dynamic. New applications get deployed. Infrastructure changes. Business processes evolve. Your perfectly-tuned detection logic becomes noisy six months later.
2. Context is customer-specific. What's malicious behavior in one environment is routine maintenance in another. Generic detection logic can't capture that nuance.
3. Expertise is scarce and isolated with individuals. The people who know how to distinguish true positives from false positives are overwhelmed. Their knowledge never gets captured in a way that scales.
Why We're Losing Good Analysts: The Training and Retention Catastrophe
Let's be blunt about what this Reddit post represents: this is how we burn out and lose security talent.
The analyst says it explicitly: "I feel like I'm stuck in an endless loop of closing the same false positives every day." This is learned helplessness setting in. This is the point where skilled people decide that cybersecurity isn't for them.
And here's the cruel irony: this analyst is learning almost nothing useful to make progress in their career.
They're not learning to investigate real threats. They're not building expertise in attack patterns. They're not developing the analytical skills that would prepare them for L2 work. They're learning to click "Close" on false positives as quickly as possible to keep the queue manageable.
One commenter who understood this dynamic suggested: "Do not ask another L1. If there is no answer from higher up you can either try to get an SOP implemented or you can quit."
That's not cynicism—that's realism. If the organization won't invest in proper structure, process, and knowledge transfer, talented analysts should leave. And they do. Constantly.
The SOC manager who said their mature MSSP gets 5k alerts per month instead of thousands per day? They can retain talent. They can develop expertise. They can build careers.
The chaotic MSSP drowning in false positives? They're churning through junior analysts every 12-18 months and wondering why they can't hire enough "qualified" candidates.
The Maturity Model Disconnect
Several commenters referenced maturity models and structured approaches. One mentioned CMMI 3.0. Another talked about having detection engineering teams and formal tuning processes.
But here's the fundamental gap: most SOCs never reach operational maturity because they don't solve the knowledge capture problem.
Mature SOCs don't just have better tools or more headcount. They have:
- Repeatable investigative processes that junior analysts can execute
- Captured expert knowledge in playbooks, automated workflows, and enrichment logic
- Continuous improvement cycles where analyst feedback improves detection logic
- Clear escalation paths based on investigation findings, not analyst uncertainty
The immature SOC in this Reddit thread has none of that. And without it, they can't grow. They're stuck in a loop of:
1. Hire L1 analysts
2. Drown them in false positives
3. Watch them burn out and leave
4. Hire new L1 analysts
5. Repeat
The Command Zero Perspective: What This Thread Tells Us About the Solution Space
Now let me connect this to what we're building at Command Zero, because this Reddit thread is essentially a detailed description of the problem we're solving.
Knowledge Capture as the Foundation
That commenter who talked about learning "the habits and personalities of different software companies" and skimming reports on quiet days? They were describing exactly what Custom Questions captures and scales.
When a senior analyst investigates an alert and determines it's a false positive, they're executing a complex decision tree:
- They check specific process lineages
- They verify certain registry keys or file paths
- They correlate against known-good behavior patterns
- They consider environmental context
- They make a judgment call based on years of experience
This expertise should be captured. It should become an executable workflow that L1 analysts can run. It should include the reasoning so that analysts learn while they investigate.
Instead, in traditional SOCs, this knowledge dies with each investigation. The senior analyst marks it closed and moves on. The L1 analyst never learns why. The same false positive appears tomorrow, and someone has to investigate it again from scratch.
The Customer-Specific Context Problem
Multiple commenters mentioned that each MSSP customer is unique and generates unique false positive patterns. This is exactly right.
But here's where traditional approaches fail: organizations try to solve this with static playbooks or generic tuning guides. It doesn't scale.
What actually works is capturing the investigative methodology that accounts for customer-specific context. When you know that Customer A runs specific backup software and Customer B uses different patch management tools, your investigations should automatically incorporate that context.
Custom Questions allows this. A question like "Is this PowerShell execution related to known administrative tools?" can pull customer-specific data about approved software, expected process behaviors, and historical patterns. The same investigative logic works across customers, but adapts to their specific environment.
The Tier Structure Fix
The Reddit thread exposed how broken tier structures are in practice. Knowledge isn't flowing from L3 to L1. Senior analysts are bottlenecks, not knowledge multipliers.
Here's how it should work:
When an L3 analyst or detection engineer designs a new alert rule, they should simultaneously create the Custom Question that guides investigation of that alert type. The question captures their expertise: what to check, why it matters, how to interpret results.
When L2 analysts investigate complex cases, they should codify their investigative methodology into questions that L1 analysts can execute. The knowledge flows down continuously.
L1 analysts now execute sophisticated investigations guided by captured expertise. They're not just clicking "Close" on false positives—they're learning to think like L2 analysts by following their investigative logic.
The Process and Methodology Gap
The original poster described chaos: everyone experimenting, no structured approach, no methodology. Several commenters said mature SOCs have formal tuning processes, detection engineering teams, and approval workflows.
They're both right, but they're missing something: process without captured knowledge is just bureaucracy.
You can implement formal tuning request workflows. You can require senior review of suppressions. You can create detection engineering teams.
But if every investigation still requires manually gathering context from multiple systems, if every alert requires a senior analyst to interpret results, if every tuning decision requires someone to remember what happened last time—you're just creating a more structured version of chaos.
The process needs to be supported by captured expertise that scales. That's the only way to get from "thousands of alerts per day" to "5k per month" without proportionally increasing your senior analyst headcount.
The Uncomfortable Truth This Reddit Thread Exposed: Most SOCs Are Fundamentally Broken
Let me end with the hardest truth from this Reddit discussion: Most SOCs are fundamentally broken, and the people running them either don't know or won't admit it.
The original poster asked "Is it normal in the industry?" And the answer from experienced practitioners was: "No, but it's common."
It's not normal for analysts to drown in thousands of false positives daily. It's not normal for there to be no structured tuning process. It's not normal for everyone to just "experiment" with suppressions and hope for the best.
But it is common. Devastatingly common.
Because fixing it requires acknowledging some uncomfortable realities:
1. Your SIEM deployment isn't done just because the vendor left. It needs months or years of tuning investment.
2. Your L1 analysts aren't the problem. Your lack of knowledge capture and transfer processes is the problem.
3. Adding headcount won't fix this. You'll just have more people drowning faster.
4. Your senior analysts' knowledge is your most valuable asset, and right now it's evaporating with every closed ticket.
5. The tier structure you have on paper doesn't reflect how knowledge actually flows (or doesn't flow) in your organization.
The organizations that figure this out will retain talent, scale expertise, and actually improve their security posture. The ones that don't will keep churning through junior analysts while wondering why their false positive rate never improves.
That Reddit post—one frustrated L1 analyst asking if their chaos is normal—should be a wake-up call for every SOC leader. The question isn't whether this is normal. The question is: What are you doing to fix it?
Steps We Need to Take to Fix the SOC
I started my career as a systems operator, evolved through network administration, and spent decades in tier-3 operations and red teaming. I've seen security operations from every angle. And here's what I know:
The false positive crisis isn't a false positive problem. It's a knowledge problem. It's a process problem. It's a maturity problem. And most fundamentally, it's a values problem.
Organizations that value their junior analysts invest in proper tooling, captured expertise, and structured growth paths. Organizations that view L1 analysts as disposable alert-clicking machines end up exactly where that Reddit poster finds themselves: drowning, confused, and heading for burnout.
At Command Zero, we're building the infrastructure to capture, scale, and transfer expert knowledge. Because we fundamentally believe that every analyst—L1, L2, or L3—deserves tools that amplify their expertise rather than drown them in noise.
That's not just good for the analysts. It's good for security operations, good for threat detection, and ultimately good for the organizations they're protecting.
The alternative is what we see in that Reddit thread: chaos with no methodology, junior analysts stuck in endless loops, and real threats getting missed because everyone's too busy clicking "Close" on false positives.
We can do better. We must do better as a community.
Because the next major breach won't announce itself with a perfectly-tuned, high-fidelity alert. It'll slip through in the noise—dismissed by an exhausted L1 analyst who's already closed 200 false positives that shift and simply can't distinguish the 201st.
That's the real cost of this crisis to our community and to every organization. And that's why solving it matters.
