Aug 29, 2023 | Content, Practice Management
The practice of “Just In Time” report publishing has emerged in the market research reports business over the past few years. The providers will advertise reports with titles loaded with keywords for SEO purposes and descriptions created via AI tools. The problem is that the reports do not actually exist.
For market research buyers the product they may order is rapidly assembled from previously issued reports and updated with content sourced via web scrapers and bots. It would be one thing if providers had significant amounts of quality content and forecasting that could be drawn from to issue a customized inquiry. It is quite another when the provider’s content is of lower quality and lesser value. The key thing to consider is the provider. A large-scale firm with significant market presence, prestige and a track record can provide much greater value and less risk than the research factories operating offshore.
So how can you tell if the product is real or waiting to be assembled?
A sample should be readily available, with content more substantive than boilerplate, no blanked-out pages and something that shows what might be available. If the provider cannot show an actual sample, it likely means the report needs to be assembled.
If there are several reports available within their coverage sector(s) (often long tail) with several keywords and phrases in the titles, the provider is likely engaging in factory practices.
Tables of contents and descriptions look similar, with minor variations within sections.
The provider claims to have several hundred reports available that are all “recently updated” Impossible given the economics and market conditions. Sectors like PC, semiconductor or consumer electronics can be updated quarterly but that product is more obvious to the buyer.
The offer of X% customized for prospective buyers? Factory assembly job
No name analysts listed on the reports or on the site.
If there are analysts associated with the product, how many have they produced within a certain time? Analysts can reasonably issue 700 pages of content per year in addition to the normal research functions they engage in. A high number of reports with 150 pages is a red flag.
Buyer Advice
While this is not an exhaustive list, it should present buyers with some things to consider. Corporate buyers in charge of sourcing data and content owe their internal stakeholders the best decision support materials available. Individual buyers may not be aware of what the industry practices are and should take note of the items listed above.
Stakeholders should insist on a bio of the analyst(s), talk to them for 15 minutes, ask some hard questions on availability of the product, ask them to explain how they can produce what they claim to have available in such timeframes. Talk to the analyst to see if they have any expertise in the space as well. Market research is fine but if they aren’t subject experts, why bother?
Market reports used to be a great way for companies to access forecasts, competitive intelligence, market feedback and analyst takes. As market research factories have grown in scale and market presence, the value of reports has declined due to the practices employed within these shops. Buyers are left to sift through piles of poor-quality products and do not necessarily understand the hidden risks.
A little awareness is a good thing….
Aug 29, 2023 | Content, Practice Management
New technology is where the action is. A lot of excitement, money, endless potential for innovation, disruption, and money. The perfect place for the Optimist to do their trade.
The Optimist gets paid for the sunshine they spread, not the forecasts of risk, negativity about companies’ downsides or alternatives that don’t support what they are advocating for. The analyst isn’t attached to the future outcomes in the way the Optimist is. The analyst is more concerned with their professional reputation.
There isn’t anything wrong with analysts being optimistic about market opportunities presented by new technology introductions or advancements. If there is a need, a means, and a viable business case then by all means, be bullish on the future.
But, optimists also never seem to the learn the lessons that the Gartner Hype Cycle teaches (https://www.linkedin.com/pulse/8-lessons-from-20-years-hype-cycles-michael-mullany/) They are reliable for getting on or in front of the next big thing, whatever that is and having a track record of issuing hockey stick/exponential forecasts and market outlooks that that ignore factors or historical frames of reference.
Red Flags:
Market reports or forecasts where everything grows and with a decided major ramp in 3 years and the market quintuples in value by year? If this happens over time you are dealing with an optimist. Hard pass if you are smart.
No negative comments or controversial takes or tries not offending anyone that might buy their stuff or they have bought in hard and won’t point out the risks?
When the quarter is down or there is a pattern of decline, do they step back from previous comments and offer new guidance or do they double down on the rebound that is coming? Do their comments resemble word salad to hide the bad news?
They produce vendor events that sell the dream vs the more objective reality of the market. The more optimism and future riches they profess the less you should trust them.
If they sell vendor advertising on their analyst platform, that is a clear sign they are not really in the analyst business. If the firm is aligned with a media outlet, be cautious as well, given the business considerations present. In addition to advertising, media platforms sell sponsored content. Pay to play isn’t good for independence.
On the other hand, if they offer paywall commentary for the good stuff then take that as a sign that there may be some objectivity possible. A buyer should be able to sort out what is there though.
If they offer advisory services, then the person or firm is most likely going to be an analyst as they must be able to push back on bad ideas or strategies or steer a client in a more beneficial direction. The optimist is far too risky to work with as an advisor as they only see the upside for vested interests or their having bought in on the vendor and industry press hype.
When you engage the Optimist, they tend to stumble, deflect or even gaslight to some extent when presented with contrarian views or are forced to explain their own when challenged. They will not own their hype tendencies.
The analyst, on the other hand, has a POV they can support and will adopt new thinking when presented with new evidence. They will continue to dig and look for ways to rebut their own analysis to make it more sound and provide value to clients and the broader market.
Aug 29, 2023 | Content, Practice Management
The arrival of LLMs such as ChatGPT, Bard and the ever-growing universe of AI tools that emerge daily, has stirred conversations as to their impact on industry sectors, business processes, people and more. This article discusses how the widespread usage of the technologies will impact the market research and report space.
AI dominates the business and social media content news landscape. Every day we see stories of new SaaS apps becoming available, advances in ChatGPT prompts, Google, Meta. Amazon, Microsoft, and others engaging in an arms race, and how Generative AI is being integrated into anything and everything. Hard to keep up, isn’t it?
The upsides are there. Productivity gains, more output, more automation, and new use cases that are pushing the limits and driving innovation. Tools that can search, summarize, rewrite, “analyze”, spot trends and even create reports for users. If the user understands the limitations of the tools and how they search, source and access data the risks are minimal.
One of the major keys to the ensuring best outcomes is via data observability (https://www.decube.io/post/data-observability-llm-accuracy) which is critical within within the analysis realm as the models only work when one can verify data sources, collection methods and accuracy. Layering AI on top of a defined data reservoir makes a lot of sense. Utilizing AI chatbots to support research and analysis is another story though.
Market research factories that make their money assembling reports using data scrapers or who engage in copy pasta are now turning to AI tools to support their “Just In Time” production model of posting reports on websites (their own or via aggregators) and backfilling the orders. AI tools are more of a means to spitting out end products that are low quality, lacking in context and worse, inaccurate given the issue of LLM hallucinations where the ChatGPT, Bard etc. provide inaccurate results.
The Just In Time model is highly problematic as the factories will compile 200 plus page documents in under 3 days packaging together previously created forecasts based on data that could very well be AI impacted along with data and content collected, synthesized and summarized utilizing these tools. And with the goal of having a report available for just about any subject or segment it is impossible for report factories to avoid these practices.
It also isn’t possible for a firm to produce 800 reports a year despite what they list on their website. The economics simply don’t prove out. A production output at that level normally requires 200 people in research, forecasting and analysis, especially if the firm sells 200-page reports. Add in the sales, marketing, production, management and overhead and even with offshore labor benefits the company must generate nearly $8 million a year in revenue to break even.
AI tools are a way for them to shave costs while giving the illusion of extensive product offering and market coverage via a team of analysts that are not obvious to anyone.
Many companies buy reports for 3rd party validation of market sizing, investment justifications, internal funding, M&A, or market perspectives. Given the issues with AI and the low standards employed by market research factories, how can a buyer or internal user feel confident in the content or numbers produced by factories or even know what they are getting though?
The answer…
Work with established research brands where the buyer is able understand the vendor’s database, research methodology and the analysts writing the report. Quality firms can produce the analysts and present their processes in an obvious way, factories cannot.
Buy directly from reputable brands and avoid the factories and the resellers who carry these products. While I am aware that some companies do screen the publishers, if the products sell then they stay in their catalogues.
If the report is not available when the buyer inquires, don’t wait 3-5 days for a scrape and compilation job. That should tell the buyer the product is a factory job.
Master the AI tools internally and not only make sure that the data and content sources are up to user standard but do your part to avoid supporting firms that lessen the value that good firms provide.
Feb 8, 2022 | Uncategorized
Analyst Relations professionals, or those handling the function, play a significant role in supporting their company’s marketing, sales, communications and strategic functions. To the extent that they can successfully make the most of their analyst relationships, their companies can derive tremendous benefits.
AR people are either nodding their heads, standing up and saying “preach!” or shrugging their shoulders as if this is not news to them. However, the stated goals of AR do not always meet the realities of how companies practice it. And the disconnect makes the engagements with analysts that much less productive or worse.
To construct this article, I drew upon my lengthy history in the analyst space which has entailed engaging both from research and business development perspectives. I also conducted over a dozen telephone interviews with analysts, practice heads and senior salespeople across a full spectrum of firms in terms of size, scope and focus. The topics focused on the following:
What elements of Analyst Relations were effective and which ones were not?
How vendors were missing opportunities to better leverage their offerings?
Some of the commercial aspects and challenges of the relationship that arise between AR and analyst firms.
The dominant themes that arose from the discussions are presented below.
Practices:
There seems to be an excessive vendor focus on Gartner’s MQ rankings and other metrics like Forrester’s Wave etc. Putting aside the segmentation games that analysts play and the pressured marketing and sales practices employed, waving the MQ about is a marketing play and is not what drives deals with customers despite what surveys conducted by these very same firms intimate. These metrics also do not necessarily impress other analysts that vendors want to impress. There are other sources for customers to collect insights and preferences from via peer reviews, user groups crowd actions and other research avenues.
Some companies properly treat AR as a strategic function and others place the role within corporate communications. Each area fulfills important roles for companies, but they have distinctly different objectives. Marcom is messaging, narrative and yes, spinning. Putting AR in the Marcom domain tends to lead to more friction between the vendors and the analysts. Marcom has a tendency to prefer having analysts act as part of the company’s marketing and messaging channel. Quality analysts are quite put off by this.
Analysts value having a relationship with AR people who get them the information they need and connect them with the right people inside the company. When AR plays gatekeeper too often and makes getting information difficult analysts are prone to bypassing the function altogether. Inevitable that analysts will create relationships they will leverage but AR does itself no favors by not being responsive or not providing access.
One additional point that was raised was in AR not necessarily having goal alignment with internal stakeholders. What problems exist from a strategic or tactical perspective that firms can support? If AR isn’t aligned with supporting the company fully or limits its focus to improper metrics, it isn’t maximizing its own value. (insert https://www.starsight.biz/2021/10/14/how-are-your-analyst-research-subscriptions-like-a-gym-membership/)
Good analysts are open to being influenced by vendors, but they want qualified data. Approaching them with unsubstantiated claims or with no means to verify leaves good analysts skeptical. Expecting analysts to accept the narrative turns them off and even more so when the business side of the equation is leveraged. This last action does damage to the relationship and whatever influence or mindshare they were hoping to gain.
Candor and professional respect play much better with them.
Regular engagements vs ad hoc or situational outreach would produce better results. When there is a certain cadence to interactions that the parties follow it makes the flow of information to analysts much smoother and potentially reduced the negative responses and surprises that a vendor may encounter. Analysts have multiple clients and companies that they follow. It becomes impossible for them to drop what they are doing and needing to suddenly get up to speed on a vendor’s initiative.
If a company operates within multiple verticals or segments with multiple analysts operating within them then it can become too challenging to engage everyone. Nonetheless, activity, strategy and considerations still need to be communicated to the market as the analyst community will still likely be writing about companies or making media comments. If AR is not going to engage personally then be sure to make it possible for analysts to follow along with what the vendor is doing. Find ways to provide information that analysts can use.
Opportunities:
Some AR programs and professionals are very heavily invested in influence and generating sales leads and give less attention to advisory, messaging and data that firms offer. They also forget, or ignore, that many analysts have real world experience in leading product groups, GTM efforts, have created technologies or sold them. Their insights can be useful if engaged properly, especially in the initial stages of a product, strategic initiative or launch plan.
It is understandable that companies have significant investments in products, strategies, and messaging. Founders have vision, leaders have targets to hit, and marketing needs to sell the story. Hearing that the strategy or implementation is a miss is hard to hear for some. But engaging analysts that tell leadership what it wants to hear is self-serving and destructive. Bringing internal stakeholders to the table to listen and consider what quality firms and the objective analysts have to say is how to best add value to a firm.
Question that arises is it the fault of the company’s AR program for shielding the company from criticism? Or is it a reflection of company culture where there is no value placed on outside thinking or constructive criticism?
Companies can be too myopic by focusing on a select few firms and missing other valuable thought leaders and potential influencers. Good people work outside the big 4 of Gartner, IDC, Forrester and Omdia. Talented people leave large firms and go to work in boutique firms, start their own or go independent. Start-ups may grow in both size and importance and wind up absorbed into larger firms. Or they may become leaders in their space. Some firms have their own events, media platforms or excel when it comes to promoting their own research and brands. Some have major presence in the trade media a company’s customers look to for news and trends.
Business:
Analysts are time constrained with the variety of tasks they are required to perform. They cannot take briefings from every company that reaches out and surprise, clients are generally going to be prioritized. Having a business relationship in place is wise in order to gain access and maximizing engagement opportunities available from the service agreements and programs works to further increase mindshare within firms.
If an analyst is important enough to reach out to for briefings, then they or their firm are likely to believe a vendor is a candidate for their programs. Briefings may very well be treated as a sales opportunity either during the time or after the fact. The P/L practices and accountabilities of the particular firm as well as the analysts themselves will vary. Professional firms do not sell opinions, but they are not acting as not for profit entities either.
Be honest and authentic about the interest in firm. Analysts understand that not everyone they talk to is going to be a client but still likely that their sales colleagues are going to call on a vendor or continue to look to grow the relationship. This should not be awkward or a surprise. It is unwise to tell an analyst there is interest then ignore the sales call from their colleague. It will not likely impact their assessment of a company, but it does not help a vendor either.
Holding a business relationship over the head of an analyst is a really bad practice. No one respects the firms that sell out their objectivity, but analysts and their firms generally take umbrage with being asked to change their opinions or beliefs based on who is buying or who won’t buy if they don’t.
The analyst business is highly nuanced with a broad ecosystem of providers and thinkers with a variety of skillsets. Companies should periodically consider if they are getting the full value available to them. There are numerous metrics that AR can apply to analyst relationships but how successful it is in challenging itself and being open to reexamining and considering new possibilities will go far in determining its ultimate value to a
company.
info@ilumatech.com