How to Evaluate If Your Agency Is Actually Working: Practical Guide with Checklist

The problem nobody wants to face

You hired a marketing agency. You pay every month. You receive reports with colorful charts. On the phone, they tell you things are "going well" and that "the numbers are growing." But at year's end, when you look at revenue, something doesn't add up.

You're not alone. According to a HubSpot study, 61% of marketers are unable to demonstrate the ROI of their marketing activities. And if marketers themselves struggle to measure results, imagine a business owner who isn't a marketing expert.

The point isn't that all agencies are incompetent. The point is that without a structured framework for evaluating performance, the client-agency relationship relies on feelings, not facts. And feelings, when there are invoices to pay, aren't enough.

The key question: if your agency vanished tomorrow morning, would you be able to quantify exactly what you'd lose? If the answer is "I wouldn't know," you have a measurement problem before you have a performance problem.

This guide gives you 15 concrete indicators — grouped into three areas — to objectively evaluate whether your agency is producing real value. Plus a scoring matrix, a guide to the "performance conversation," and criteria for deciding whether to give more time or make a change.

Area 1: Performance — the numbers matter

The first 5 indicators focus on measurable results. They're the most objective and the hardest to manipulate — if you know where to look.

Indicator 1: Qualified traffic growth

Not total traffic — qualified traffic. A 200% increase in visits that don't convert is worth nothing. What matters is traffic coming from the right sources (organic, paid, strategic referral), landing on the right pages (products, services, contact), and exhibiting the right behavior (time on page, pages per session, conversions).

How to measure: Google Analytics 4 > segment by source/medium, filter by commercial landing pages, compare periods (month over month, quarter over quarter). A serious agency must be able to show this data segmented, not aggregated.

Benchmark: 15-25% year-over-year organic growth is good for most industries. For paid traffic, look at the ratio between spend and qualified visits (not total ones).

Indicator 2: Conversion rate

The conversion rate is the percentage of visitors who take the desired action: fill out a form, call, purchase, subscribe. It's the metric that connects traffic to business results.

How to measure: define macro conversions (purchase, quote request) and micro conversions (download, newsletter sign-up). Track both by traffic source. The conversion rate should improve over time, not just the volume.

Benchmark: e-commerce 1-3%, B2B lead generation 2-5%, dedicated landing pages 5-15%. If your numbers are significantly below the industry average, the agency needs to explain why and have a plan to improve them.

Indicator 3: ROAS (Return On Ad Spend)

For every euro invested in advertising, how many euros in revenue do you generate? ROAS is the king metric of performance marketing and must be tracked by channel, by campaign, and over time.

How to measure: (Revenue generated by campaign) / (Ad spend). Note: ROAS includes only media spend, not the agency fee. For the full picture, also calculate total ROI including all costs.

Benchmark: 3-5x ROAS is considered good for most industries. Below 2x, in most cases you're not covering costs (considering margins, agency fee, and operational costs). Above 8x, verify that tracking is correct — it might be overattributing conversions.

Indicator 4: Lead quality

Generating 100 leads per month is useless if only 3 are genuinely interested and qualified. Lead quality is often more important than quantity, especially in B2B where a single contract can be worth tens of thousands of euros.

How to measure: define what makes a lead "qualified" (budget, authority, need, timing — the BANT framework). Track the conversion rate from lead to opportunity and from opportunity to customer. If the agency generates leads but the sales team systematically discards them, there's a targeting problem.

Benchmark: a qualification rate (lead > opportunity) of 20-30% is acceptable. Below 10%, leads aren't being targeted correctly.

Indicator 5: Customer Acquisition Cost (CAC)

How much does it cost to acquire a new customer, considering all marketing costs (agency fee + media spend + tools + internal time)? The CAC must be sustainable relative to the customer's lifetime value (LTV).

How to measure: (Total marketing and sales costs) / (New customers acquired in the period). The LTV:CAC ratio should be at least 3:1 for a sustainable business.

Benchmark: varies enormously by industry. E-commerce: 30-100 EUR. B2B SaaS: 200-1,000 EUR. Professional services: 500-5,000 EUR. What matters most is that the trend is improving.

Area 2: Process — how the agency works

Numbers don't tell the whole story. An agency can have good short-term results but with an unsustainable process. These 5 indicators evaluate the how, not just the what.

Indicator 6: Reporting frequency and quality

A monthly report is the bare minimum. But frequency isn't everything: quality matters. A good report isn't a Google Analytics data dump. It's a document that says: what happened, why it happened, and what we'll do differently next month.

What to look for: analysis, not just numbers. Actionable insights, not just charts. Comparison with objectives, not just with the previous month. If the report you receive could be automatically generated by a tool, the agency isn't adding intellectual value.

Indicator 7: Response time

How long does the agency take to respond to your emails? To handle an emergency? To implement an urgent change? Responsiveness is an indicator of how important you are to them.

Benchmark: emails within 24 business hours (preferably within 4). Emergencies within 2-4 hours. Non-urgent changes within the week. If you regularly wait 3-4 days for a response, you're probably not a priority.

Indicator 8: Proactivity

Does the agency propose new ideas without being asked? Flag opportunities, trends, risks? Bring market data and insights? A proactive agency is a strategic partner. An agency that waits for your instructions is an executor — and for an executor, you're paying too much.

What to look for: at least one unsolicited proposal per month. Flagging of relevant changes (algorithm updates, new platform features, competitor moves). Ideas for tests and optimizations.

Indicator 9: Transparency

Does the agency show you where every euro of your budget goes? Tell you when something didn't work? Alert you when a project is going out of scope or over budget? Transparency is the foundation of trust, and trust is the foundation of every lasting professional relationship.

Red flags: reports that show only positive data. Reluctance to share account access. Invoices without activity breakdown. Vague answers when you ask "where did my money go."

Indicator 10: Deadline adherence

Are agreed deadlines met? Are projects delivered on time? If there are delays, are they communicated in advance with an explanation and a recovery plan?

Benchmark: an 80-90% deadline adherence rate is realistic (unexpected things happen). Below 70%, there's a project management problem. If deadlines are systematically missed without advance communication, it's a serious red flag.

Area 3: Relationship — the strategic value

An agency can have good numbers and a good process, but if it doesn't add strategic value, you're buying a commodity service that could be replaced by anyone — or by an automated tool.

Indicator 11: Strategic thinking

Does the agency think in terms of business strategy or just marketing tactics? Does it understand your business objectives or does it just "manage campaigns"? A strategic agency connects every activity to a business outcome and can tell you why one action is more important than another.

What to look for: recommendations based on data and market insights, not personal preferences. The ability to say "no" when a client idea isn't strategically sound. A long-term vision, not just quick wins.

Indicator 12: Knowledge of your business

After 6 months of collaboration, does the agency know your market, your customers, and your competitors? Can it explain what differentiates you? Does it understand your industry dynamics? An agency that still doesn't understand your business after months hasn't invested time in the relationship.

Practical test: ask the account manager to describe your ideal customer and your main competitive advantage. If the answer is generic or wrong, the agency doesn't know you well enough.

Indicator 13: Team stability

How many times has your account manager changed in the past year? How many people from the original team are still working on your project? High staff turnover is one of the most underrated problems in the agency relationship: every personnel change means lost knowledge, onboarding time, and strategic discontinuity.

Benchmark: one account manager change per year is acceptable. Two or more changes indicate a structural problem at the agency (undercompensation, overload, toxic culture).

Indicator 14: Adaptability

The market changes, your priorities change, platforms change. Does the agency adapt quickly or does it stick to the "original plan" even when conditions have shifted? A rigid agency becomes a brake, not an accelerator.

What to look for: willingness to revise the strategy when data suggests it. Ability to reallocate budget between channels based on performance. Adoption of new platforms and formats when relevant to your target audience.

Indicator 15: Educational value

Does the agency make you more knowledgeable over time? Does it explain the why behind decisions, not just the what? Does it share knowledge or keep it to itself (to make you dependent)? An excellent agency wants an informed client, because an informed client makes better decisions and produces better briefs.

What to look for: clear explanations in reports, training on tools, sharing resources and trends, transparency about decision-making logic.

The scoring matrix: evaluate your agency

Use this matrix to assign a score from 1 to 5 for each indicator. Be honest and, where possible, base your judgment on objective data.

How to interpret the score

• 60-75 points: your agency is doing excellent work. Invest in the relationship
• 45-59 points: good performance with areas for improvement. Schedule a constructive conversation
• 30-44 points: critical areas. An improvement plan with clear timeline and consequences is needed
• 15-29 points: the relationship isn't working. Seriously consider making a change

The 6 red flags that indicate it's not working

Beyond the scoring matrix, there are specific signals that — on their own — should set off alarm bells.

Red flag 1: Identical reports month after month

If the May report is practically identical to the March report — same phrases, same structure, same "recommendations" — the agency is using a template and not analyzing anything. A report should reflect what happened that specific month, with specific insights.

Red flag 2: No tests underway

If you ask "what A/B tests are you running?" and the answer is vague or nonexistent, the agency isn't optimizing. Continuous testing is the heart of performance marketing. Without tests, you're paying for maintaining the status quo.

Red flag 3: Defensiveness when you ask about results

An agency confident in its work welcomes questions about performance. If the reaction to "how are we doing?" is defensive, evasive, or irritated, there's a problem. Either the results aren't there, or the agency can't communicate them — and neither is acceptable.

Red flag 4: Vanity metrics instead of business metrics

"We reached 500,000 impressions!" "Followers grew by 40%!" If the report is full of vanity metrics (impressions, reach, followers) and lacking in business metrics (conversions, leads, revenue), the agency is masking the absence of concrete results with numbers that sound good but don't pay the bills.

Red flag 5: Dependence on the agency to access your data

If you need to ask the agency for a report because you don't have direct access to the data, you have a governance problem before you have a performance problem. Your data must be yours — accessible at any time, from any device, without asking anyone's permission.

Red flag 6: The agency doesn't know your industry after months of collaboration

If after 6 months the account manager still confuses your products, doesn't know your main competitors, or doesn't understand your market dynamics, the onboarding investment wasn't made. And if they don't invest time to understand you, they can't produce strategic work.

How to have the "performance conversation"

You've done the evaluation. The score isn't great. Now you need to talk to the agency. Here's how to do it constructively.

Prepare the data

Don't show up with feelings. Present data: "ROAS has dropped 20% over the last 3 months," "Average response time was 3.5 days last quarter," "We received 45 leads but only 3 were qualified." Data eliminates interpretations and focuses the conversation.

Distinguish between problems and preferences

A declining ROAS is a problem. Not liking the banner color is a preference. The performance conversation must be about problems, not preferences. If you mix the two, the agency won't understand what's truly critical.

Ask for a plan, not justifications

The question isn't "why are results poor?" but "what will we change in the next 90 days to improve?" A serious agency responds with a specific action plan, with responsibilities and timelines. If they respond with excuses and generic promises, they probably don't have a plan.

Define an improvement timeline

After the conversation, agree on specific objectives for the next 90 days. Put them in writing. Schedule reviews at 30, 60, and 90 days. If at 90 days there's no significant improvement, you have an objective basis for your decision.

When to give more time vs. when to make a change

Give more time when:

• The agency acknowledges the problems and presents a credible plan
• Problems are partially caused by external factors (seasonality, platform changes, economic context)
• The collaboration is recent (less than 6 months) and the agency hasn't had sufficient time
• The team is competent but the process needs improvement
• Results are improving, even if not yet at target level

Make a change when:

• After 90 days from the improvement plan, there's no progress
• The agency is defensive, doesn't acknowledge problems, and doesn't propose solutions
• The team changes constantly and nobody knows your project
• You don't have access to your data and the agency resists transparency
• Results are worsening and the agency has no explanations or plans
• Trust is compromised (e.g., you've discovered hidden costs or manipulated data)

The 90-day rule: if after a frank conversation and an agreed action plan you don't see measurable improvement in 90 days, the probability that the situation will improve spontaneously is very low. It's not a matter of patience — it's a matter of data.

How to manage the transition

If you decide to change agencies, the transition is a delicate moment. Here are the best practices.

• Give contractual notice: respect the terms, even if you're dissatisfied
• Request a handover: campaign documentation, access to all accounts, activity history
• Verify asset ownership: advertising accounts, domains, source files, content
• Plan an overlap: if possible, have the new agency work in parallel for 2-4 weeks during the transition
• Don't burn bridges: honest and respectful feedback closes the relationship professionally and leaves the door open for potential future collaboration

Frequently asked questions

How long before I can realistically evaluate an agency?

It depends on the type of service. For performance marketing (Google Ads, Meta Ads), 3 months is enough to see significant trends. For SEO, 6-12 months are needed. For branding, results are more qualitative and require different metrics (brand awareness surveys, sentiment analysis). In general, 6 months is a reasonable period for a first structured evaluation.

Should I share the scoring matrix with the agency?

Yes, it's very helpful. Transparency builds trust and gives the agency the opportunity to improve in critical areas. A mature agency will appreciate structured feedback. If the agency reacts badly to an honest, data-based evaluation, that's a red flag in itself.

How do I distinguish between a poor agency and overly ambitious targets?

Look at industry benchmarks. If your ROAS target is 10x in an industry where the average is 3x, the problem might be the expectation. If your ROAS is 1.5x in an industry where the average is 4x, the problem is likely the agency. Always compare against market data, not your subjective expectations.

The agency says results are good but I see no impact on revenue. Who's right?

Potentially both. Marketing results (traffic, leads, engagement) can be good but fail to translate into revenue due to problems in the sales funnel (e.g., the sales team doesn't follow up on leads, the website converts poorly, pricing isn't competitive). Track the entire journey from first contact to sale to identify where the chain breaks.

Can I evaluate an agency if I don't have access to the data?

No, and that is itself the first problem to solve. Without direct access to data, any evaluation relies on what the agency chooses to show you — which may be selective. Immediately request admin access to all accounts and platforms.

How much does it cost to change agencies?

Direct costs include: potential contract penalties, the new agency's onboarding fee (typically 1-2 months), possible performance dip during transition (1-3 months). Indirect costs include the internal time for selection and onboarding. In total, budget for 2-4 months of "additional cost." This is why it's important to choose well the first time — and to evaluate methodically before deciding.

Is there a better time of year to change agencies?

Avoid changing during your business's peak periods (e.g., before Christmas for e-commerce, before summer season for tourism). The ideal time is right after a peak period, when you have 3-4 "quiet" months for onboarding and optimization before the next peak.

Sources and further reading

• HubSpot — State of Marketing Report, 2025
• Gartner — Marketing Organization Survey: How CMOs Are Restructuring for Growth
• Clutch — How Businesses Select and Evaluate Agencies, 2024
• AgencyAnalytics — Agency Benchmarks Report, 2025
• Forrester — The Agency Performance Evaluation Framework
• Marketing Week — "How to Know When It's Time to Change Agency", 2024