Myth 1 : Hot Takes Drive Sales
[00:00] Jenn Gile: the first myth that we're going to tackle is hot takes are what sells. And uh where this myth comes from is uh definitely there's a perception um that analysts get paid to make up categories to make up buzzwords. Um you know if we look uh over the last few years whether you think they're real or not there's categories like ASPM which we can talk about today um uh shift left I think may have come from an analyst at some point. Um you know what what would you say about what this myth means and where the truth in it lies and where perhaps it's busted?
[00:41] Katie Norton: yeah Um you know I think there's obviously a incentive right in the tech industry to coin new trends and categories to be the the the person whether you're an analyst or not to excuse me have created a trend or a category. As much as you know we're currently in an acronym suit most of the time, uh I the way I look at it is that an analyst's job really is to interpret the market not manufacture it.
And you know I think an important thing to understand about working with analysts is our job is to help you make sense of like where a company or a vendor fits into broader trends, to understand how a market is likely to evolve and you know what we're seeing across vendors, users and and you know the data that we collect. And that doesn't mean that like hot takes are wrong. I mean I have plenty of them, but I don't think that it's the MO for every analyst um or every firm right, and I'm personally not a big fan of acronym making, I have not made an acronym or a category in my time as an analyst just like for its own sake. Um I think maybe what...
A Day in the Life of an Analyst
[02:06] Katie Norton: will help a little bit is to talk about what it is I actually do on a day-to-day basis. That you know in my day I'm often spending my time either I take briefings from vendors. So I cover application security and software supply chain security. So vendors in that space they meet with me on a fairly regular basis. They walk me through their products, new features, their road mapaps. Um I also take inquiry calls from vendors too who want feedback. Um they look to me to test their messaging, their strategy, their or they might want say competitive or market intelligence.
But I also speak with not just vendors, I talk with end users. Um you know I'll have calls where a a large enterprise will come to me and say you know we're shortlisting a SAS tool and we want like an outside neutral perspective on uh you know this is our list, Are we missing anybody that maybe there's somebody in the market they haven't considered yet? And I help work through their maybe list of priorities and help them align them with what I know from all the briefings that I do about what a vendor offers.
Um I also talk to investors who want to understand where a market segment is headed. So like bankers and folks that are you know putting the money into the market right, they they look to me to understand larger trends and themes and then all of this comes together in that I write research. I have what's called a research practice, I put together reports, I get to conduct conduct surveys annually, uh and then I go to like conferences, I speak on panels and in webinars like this and I try to stay active and engaged with you know practitioners and vendors on social media.
And so all of that you know at the end of the day I think in terms of the busting while I definitely think I'd say this was is plausible if we're thinking about the show right, like there are plenty of analysts that you can blame for acronym creation but I think that is sort of a surface uh what comes out of all of the the work that we're doing to sort of be those I think I like to look at myself as like maybe a trusted advisor to a that's sort of a independent party you know for a lot of different audiences.
Myth 2: Analysts Are Paid to Play
[04:30] Jenn Gile: I think that segus really nicely into myth number two which is analysts are paid to play. And if we like unravel that a little bit how can I expect you to be a trusted advisor if you're getting paid to push a vendor so what would you say about that particular myth?
[04:54] Katie Norton: yeah I think this is probably maybe the lo most like vocal or longstanding myth about the analyst industry and I get why it persists. I mean after all analyst firms are businesses right, you can't deny that and it's so there there's always exchanges of money. Vendors do pay and to work with analysts right, but speaking which I can only do is speak for myself and for IDC right, i can't speak for all analyst firms but I would generally say this is pretty true from I have lots of friends that work at all of the competitive firms as well.
And you know the one thing I can say is that the research that I publish into my practice and the advisory that I give particularly to end users is conducted um completely independently of those things. And so I can kind of give you an example of uh I just recently did uh an IDC marketcape which is one of those vendor evaluation pieces of research and these are the things that usually drive a lot of the payforplay conversations. There's a lot of opinions right when you go and out there as an analyst and you put out like this chart and that's all anybody sees right is the little dots and where they landed and and everybody has thoughts about the correctness of that right or how did I come to those conclusions and oftentimes because of the way the analyst firm model especially for the like larger firms a lot of stuff is paywalled right or it's like deep the the methodology is buried deep in my report was like 54 pages right and the the like back of it explained how it was all done.
And but when I did that piece of research that involved um a pre-qualification survey, a um I had to set criteria, I had a 125 question RFI that they had to answer. Um each vendor that participated um had to do an hour demo and an hour briefing and they had to give me customer references that I surveyed and talked on the phone to. And then I took all of that information and it went into a very granular detailed you know rubric really and it was a very quantitative process behind that. But that stuff isn't like unless you're the analyst that's doing it that's all you see is the chart right.
So I think the thing to kind of narrow in on is that you know while analyst firms are businesses the integrity of the research is what gives that business value. If there isn't independence the whole model falls apart right. So I you know I think of all of the things I would slap busted on at least for myself right, is that you know the research that I'm doing there whether a vendor is a client of IDC's or has paid IDC any money over time influences does not influence my research in any way.
Choosing the Right Analyst Firm
[08:03] Jenn Gile: you and I have talked a little bit about um how to decide if an analyst firm is right for you know your specific use case. Um so you know you've mentioned several times that IDC and you yourself focus on research. Can you talk a little bit about the other categories, the other flavors of analyst firms that are out there as people are thinking like do I want to bounce ideas off a firm like what are the choices?
[08:35] Katie Norton: yeah Yeah Um every there there are you know a lot of times you hear like the the big three right like IDC, Gartner, Forester are probably the most well-known firms but there are uh tens, 20 like there are a lot of smaller or boutique firms um it and each one sort of has this like flavor of of what they're good at. IDBC is known for market sizing and forecasting. We model every vendor in a market and we have very um the international data company if you didn't know that's what IDC stands for. And so excuse me um our research is very um data oriented. We do a lot of surveying um that kind of stuff.
But then you have other analyst firms. Some um focus more on like say enduser advisory work. Maybe they they if you split your time between vendors and endusers they spend more time advising enterprises than they do maybe talking to vendors. Others really focus on like influence maybe social media presence um and do more say like content marketing where they'll partner you know with a vendor and develop something and have that sort of neutral or independent validation of of the work that they're doing.
Um and then also it's like some cater more towards different sizes of organizations. So you know some look more towards startups and helping them get visibility. Others maybe primarily serve like enterprise clients um and that that want like really deep dives into those like established vendors in a market. Um so a lot of times it's just a matter of figuring out what it is that you're looking to get out of an analyst relationship and then kind of seeing who's out there and matching up with them.
And also some of its personality too. Like I frequently work with people because they just like working with me right regardless of what area I'm covering, they I my boss for many years at IDC had folks that would continue to cut he was an analyst for 35 years and the connections he built across the industry like people just trusted him as as a a neutral third party regardless of what his like coverage area was what market he was researching and and then would just work with him you know over his whole course of time as an analyst.
Myth 3: Analysts Are Out of Touch
[11:10] Jenn Gile: All right we have a couple questions in the Q&A. So I'm gonna skip one of our myths and I'm gonna address the third myth and then we're going to go to the Q&A. All right Um the third myth that we're going to talk about is um analysts are out of touch. You know they're not practitioners, They're not in the job. Um why would I you know I can go call up you know my friend who works at another company, Why would I trust an analyst who who's not in the role?
[11:39] Katie Norton: yeah No it's a good question. Honestly I I a good example here is I was not an application security practitioner but before coming an analyst and kind of getting to there, you know a lot of analysts some do come from a practitioner background. That's not an uncommon path for someone to take they you know maybe have been a pra practitioner and a lot of times um I will say as a younger woman I am an anomaly in the the the analyst world oftentimes you find uh more seasoned individuals right that have been around for for a long time.
And but the the thing is is whether they were practitioners or not it doesn't necessarily mean that they're out of touch and some of the best analysts I find come from like really non-traditional backgrounds because the core skills of what we do are more analytical not necessarily operational. So like for example um you know while being a practitioner in whatever field or market you're researching is helpful because it you know it does obviously help you understand how certain technologies are implemented or maybe had experience with real world challenges.
Sometimes that can create blind spots too of like you have this view of like this is how we did it at the organization I worked at for the last 20 years and it makes it harder to step back and see how the market as a whole is evolving because being an analyst requires you to look across like hundreds of companies. I mean I alone in application security probably I want to say my like spreadsheet and like database I have built has like 800 vendors on it that and the you know myriad technologies underneath of that that I track and that broad view and the ability to identify shifts and trends that's not necessarily visible from like any single practitioner's perspective.
So and I think the other thing I'll say on this too is that like technology also changes so rapidly that even if you had hands-on experience at one point and and you would see that as valuable that can become dated itself right pretty fast. So I think um the important thing to think about analysts is it's more about that ability to ask the right questions, to connect the dots across like disperate data, to identify patterns right. It's about like I look at my skill set is that I'm really curious, I'm a critical thinker and you also have to be able to clearly communicate things because part of our job is translating like really complex and sometimes conflicting in you know information into insights that people can actually like act on.
And you know a lot of times I think what people say about analysts is we kind of have this bird's eye view right, there's a particular position we have when looking at a market that not a lot of people have. I get to talk to like people who are direct competitors with each other and talk to their customers and go to user conferences and talk to the people investing money into the market. So I kind of have all of these different perspectives. So whether I've had I've been able to learn the ins and outs of application security about the the topic and and the technology but like to do my job well it's more about the like the skill set that's brought in in my opinion. Yeah so we...
The Impact of AI on Tech Analysts
[15:19] Jenn Gile: have an interesting question um about AI and how it might be disrupting the job of a tech analyst or even the necessity for a tech analyst. So let's like maybe talk a little both like shortterm and long term because I think short term we're not seeing uh I I would not anticipate AI would replace analyst shortterm just based on you know what you just said. Um but where do you see maybe it uh supplementing or replacing?
[15:53] Katie Norton: yeah I think right now this is a good question and we ask it ourselves like in the industry uh uh and and it's something we think about right and I think not just our industry is subject to this question of like can AI I think everybody in every job is like can AI replace what I'm doing. I would say in the short term and right now I look at AI as as an advantage if you know how to use it well and not just create uh you know AI generated a as my I have eight-year-old Gen Alpha kids that are very frequently talk about AI slop and uh uh and you can if you don't know how to use generative AI well you can succumb to AI slop as an analyst as well.
But I think if you know how to integrate it in the right way into your workflow I think it actually at least for me I find it makes me faster. Like a good example is like I love Notebook LM from Google. It has made my ability to like synthesize a bunch of information so much faster than I ever could in the past. Yeah Plus that anybody who needs to like analyze uh documents calls whatever like Notebook LM is so great for that.
Yeah like for me it's like obviously with and the umbrella underneath application security and software supply chain security is huge right like trying to track everything and understand everything that's happening from like the recent npm attack to uh you know understanding MCP servers and their security weaknesses and um how AI agents work and like all this technology like using AI to help you learn I think is can help make an analyst better and faster.
Long term I don't think the question you're asking is wrong. Um I do think when I look at it in terms of like will will is it going to change the analyst industry probably. Um at least it's going to force maybe some of the older models to re and styles of like the the payw wall gated um you know some of the way the the the analyst business is run might have to adjust to adapt to AI.
But the one thing I will say is analysts have a ton of NDA information. Uh we get previews of things like well in advance of of when they're they're out there. And we also again that like that multitude of different personas in an industry or a market that we're able to speak with and connect is a way that like I don't think AI ever could get to because even the stuff even if you trained a model on like everything IDC ever wrote right there's always this level of information that's not published that we have that we're able that that comes up in conversation. So much of what we do is not even written right like it's hading conversations and advisory that that's the part that I think that that AI can't touch right.
