Stop Scrolling Through 2,000 Keywords: Let AI Make the Judgment Calls, Keep the Rules in Code

#paid-ads#python#data-analysis

A Google Ads keyword workflow for marketers who spend more time in spreadsheets than in the Ads dashboard.

If you run Google Ads at scale, you know the ritual. Export a keyword list from Keyword Planner, add a competitor tool’s export and last quarter’s brainstorm, merge them in Excel. Now there are 2,000 rows, and the scrolling starts.

“Is this one commercial? Is someone at a big company typing this, or a student? Does the product even do this? Is this person ready to buy, or just Googling a definition?”

That’s four small judgment calls per keyword. Across 2,000 keywords it’s 8,000 decisions, made at 4pm on a Thursday with the coffee wearing off. Some will be wrong. They won’t be consistent either: “enterprise” in row 40 isn’t quite the same “enterprise” as in row 1,400.

This post walks through a small workflow that replaces the scroll: an AI model called Jev makes the judgment calls, and a short script applies the picking rules. The full run takes about a minute and costs less than ten cents.


Before vs after

BEFORE                                         AFTER
──────────────────────────────────────         ──────────────────────────────────────
keyword-universe.xlsx  (2,000 rows)            keyword-universe.csv  (2,000 rows)
        │                                              │
        ▼                                              ▼
Scroll, read, judge each row by hand           Jev: 4 judgments per keyword
  intent? enterprise? fit? ready to buy?         (with probabilities, ~1 minute)
        │                                              │
        ▼                                              ▼
Colour-code cells, add filter columns          Code: hard filters + ranking score
        │                                              │
        ▼                                              ▼
Copy "good ones" into a new tab                picked-keywords.csv
        │                                        top 100, ad group, a reason on
        ▼                                        every row that didn't make it
Sort into ad groups by hand
Before (manual Excel) After (Jev + code)
Time for 2,000 keywords A day or more About a minute
Cost Your day Under $0.10
Consistency Drifts with fatigue Same definition on every row
“Why isn’t X picked?” Nobody remembers Written in a column
Rule change (“cap CPC at $25”) Redo the filtering Change one number, re-run in a second
Who decides the criteria You Still you

Meet HelpNest (the imaginary advertiser)

HelpNest sells helpdesk software to mid-size and enterprise support teams: shared inboxes, ticketing, live chat, a knowledge base, AI reply suggestions, SLA tracking, and integrations with the CRMs and e-commerce platforms those teams already use.

Just as important is what HelpNest is not: it isn’t a sales CRM, a marketing automation tool, or a project management app. That turns out to be one of the most useful things to tell the AI. More on that below.

The marketing team has a keyword universe of ~2,000 keywords and wants to pick the top 100 to launch a campaign, then sort them into ad groups.


What is Jev? (The ELI5 version)

Most people know general-purpose AI chatbots like ChatGPT or Claude. Ask a question, get a paragraph. They’re great at open-ended work: research, brainstorming, drafting.

Jev is different. It doesn’t write paragraphs. It answers a specific question with a specific type of answer, very fast.

Think of it as the difference between:

  • A consultant (general-purpose LLM): “Tell me about this keyword.” → Three thoughtful paragraphs.
  • A trained reviewer with a checklist (Jev): “Is this keyword commercial, informational, navigational or transactional?” → “Commercial, 91% sure.”

Jev is made by TypeSafe, which calls these “System One” models, after the fast, intuitive kind of thinking people do without deliberating. Reading “free helpdesk for my side project”, anyone just knows it isn’t an enterprise buyer. That’s System One, and it’s the kind of call Jev makes.

Jev gives three kinds of answers (“primitives”). This workflow uses two:

Primitive What it answers Example
Choice “Which one of these options?” Intent: Informational / Navigational / Commercial / Transactional
Noul “How likely is this to be true?” (a probability of yes from 0 to 1) “Is this an enterprise buyer?” → 0.82
Score “Where does this sit on a scale?” (not used here)

Why Jev instead of pasting the list into ChatGPT?

  1. Probabilities, not just labels. This is the big one. A chatbot says “Enterprise: Yes.” Jev says “Enterprise: 0.82.” That number makes it possible to rank, set thresholds, and spot close calls (“this one is 0.49, worth a look”). Every scoring idea in this post depends on it.
  2. Answers always fit the format. Ask for one of four intents and you get one of four intents. No “Commercial-ish, but also kind of informational…” and no broken table on row 1,312.
  3. Consistency. Re-run it and the answer is the same. Row 40 and row 1,400 are judged against the same definition.
  4. Speed and cost. ~2,000 keywords × 4 questions took about 65 seconds and cost $0.09. A chatbot would take much longer, cost more, and still need results pasted back in batches.
  5. “Why not just ask a cheap LLM for a strict format?” It’s a fair question, and the format isn’t really the advantage: any LLM can be told to return one clean label. The probability is. A rule like “skip it if enterprise is below 0.5” needs a number, and a chatbot only gives you a word (one developer’s example is exactly this: a Jev score feeding an ordinary if statement). A model trained for one kind of decision also tends to judge it better than a general chat model. Still, if a general model gives better answers on your list, use it.

General LLMs still have a job. One (Claude) handled the open-ended work: reading the product website to write a description of the company, helping design the questions, and proposing the ad-group structure. The LLM designs the checklist; Jev applies it to every row.


Separation of concerns: what goes to AI, what stays in code

The most important idea in this post, stated plainly:

If it’s a judgment, ask Jev. If it’s a rule or a number, write code.

Task Who does it Why
“What’s the intent of this search?” Jev Needs language understanding. No formula for it.
“Is this an enterprise buyer?” Jev A judgment call, the same one you’d make reading the row
“Does HelpNest actually do this?” Jev Needs to understand both the keyword and the product
“Is this searcher ready to buy?” Jev A judgment call
“Which ad group does this belong in?” Jev Matching against written definitions
“Is CPC between $0 and $30?” Code It’s math. AI is unnecessary and less reliable here.
“Does the keyword contain the brand name?” Code An exact text match
“Rank, then take the top 100” Code A sort
Changing the rules later Code Edit one line and re-run. Jev isn’t called again.

That last row matters day to day. Jev’s answers are saved to a file (jev-answers.json). When the request comes in to “take 150 keywords, not 100” or “cap CPC at $25”, one number changes and the script re-runs. It takes a second and costs nothing.

Developers outside marketing use the same split: the model makes the small, fast call (“I’m using it as a router”) and plain code does everything else. In high-stakes systems, the AI isn’t allowed to run anything directly. That keeps every step easy to test and easy to trace.


The project structure

Five files. No servers, no dashboards.

helpnest-keywords/
├── keyword-universe.csv     # The export. Read-only, never edited.
├── classify.py              # The script: asks Jev, saves answers, applies the rules
├── jev-answers.json         # Jev's answers per keyword (intent, enterprise, fit, urgency)
├── jev-adgroups.json        # Jev's ad-group answer per keyword
├── picked-keywords.csv      # The result: open it in Excel
└── log.md                   # Every rule decision, and why

Three commands:

python3 classify.py              # ask Jev the 4 questions for every keyword
python3 classify.py --adgroups   # ask Jev which ad group each keyword belongs in
python3 classify.py --pick       # apply the rules → picked-keywords.csv

The log.md file is underrated. Every rule change gets a line saying what changed and why. When someone asks “why isn’t keyword X in the campaign?” three weeks later, the answer is written down.


Step 1: The input (the keyword universe)

Nothing fancy. This is the standard Keyword Planner export:

Keyword Avg. monthly searches Top of page bid (low) Top of page bid (high) AVG CPC (mean) AVG CPC (geometric midpoint)
help desk ticketing system for enterprise 500 12.40 48.90 31 25
deskpilot alternatives 50 8.10 24.60 16 14
free helpdesk software 5,000 2.10 9.80 6 5
what is a ticketing system 5,000 1.20 6.40 4 3
crm for real estate agents 500 9.30 35.10 22 18
deskpilot login 500 0 0 0 0

(DeskPilot and SupportHive are imaginary competitors.)


Step 2: Tell Jev who the advertiser is

Jev can’t browse the web, so it gets a written description of the company. Think of it as briefing a new hire before they review the list:

ADVERTISER = (
    "HelpNest is a customer support platform for mid-size and enterprise teams. "
    "It gives support teams a shared inbox, ticketing with SLAs, live chat, a "
    "knowledge base, AI reply suggestions and customer service automation, with "
    "integrations into CRMs, e-commerce platforms and phone systems. "
    "Sold as SaaS with enterprise features: SSO, audit logs, data residency. "
    "It is not a sales CRM, marketing automation, project management or "
    "general productivity application."
)

That last sentence does real work. Without it, “crm for real estate agents” looks like a fit (“it said CRM!”). With it, Jev scores it low.


Step 3: Ask the four questions (how the classification works)

Here’s how the questions look in classify.py. Each has instructions (what to judge) and criteria (what each answer means):

QUESTIONS = {
    'intent': {
        'type': 'choice',
        'instructions': 'What is the search intent of the Google search query in `keyword`?',
        'criteria': {
            'Informational': 'The user is looking for information or an answer to a question.',
            'Navigational':  'The user is looking for a specific website, such as a named brand.',
            'Commercial':    'The user is comparing products before buying: best, top, '
                             'alternatives, vs, reviews, or a software category.',
            'Transactional': 'The user is ready to act: buy, pricing, sign up, demo, trial.',
        },
    },
    'enterprise': {
        'type': 'noul',
        'instructions': 'Is the query in `keyword` most likely typed by someone at a mid-size '
                        'or enterprise company with a large, high-value support use case?',
        'criteria': {
            'true':  'A business buyer: high ticket volumes, multiple teams, integrations, SLAs, compliance.',
            'false': 'An individual, freelancer or tiny business, a free tool, or a how-to question.',
        },
    },
    'direct_fit': {
        'type': 'noul',
        'instructions': 'Does the query in `keyword` directly match a capability of the '
                        'advertiser described in `advertiser`?',
        # criteria: helpdesk, ticketing, live chat, KB, support automation, competitor
        #           alternatives = true; sales CRM, marketing, project management = false
    },
    'urgency': {
        'type': 'noul',
        'instructions': 'Does the query in `keyword` signal the searcher is ready to adopt '
                        'or buy a solution soon, rather than still learning?',
        # criteria: tool, software, vendor, pricing, demo, alternative = true; definitions = false
    },
}

And here’s the entire “talk to Jev” function. One request per keyword, all four questions answered in parallel:

def ask(keyword, questions=QUESTIONS):
    body = json.dumps({
        'model': 'jev-1.14.0',
        'state': {'keyword': keyword, 'advertiser': ADVERTISER},
        'questions': questions,
    }).encode()
    req = urllib.request.Request(API, data=body, method='POST', headers={
        'Authorization': f"Bearer {os.environ['TYPESAFE_API_KEY']}",
        'Content-Type': 'application/json',
    })
    with urllib.request.urlopen(req, timeout=60) as r:
        return json.load(r)

The script sends 8 keywords at a time and saves answers as it goes. If it stops halfway, the next run only asks about keywords it hasn’t seen yet.

What comes back: jev-answers.json (3 records)

{
  "help desk ticketing system for enterprise": {
    "model": "jev-1.14.0",
    "answers": {
      "intent": {
        "type": "choice",
        "choice": "Commercial",
        "confidence": 0.85,
        "probabilities": { "Informational": 0.04, "Navigational": 0.0, "Commercial": 0.91, "Transactional": 0.05 }
      },
      "enterprise": { "type": "noul", "noul": 0.88 },
      "direct_fit": { "type": "noul", "noul": 0.97 },
      "urgency":    { "type": "noul", "noul": 0.70 }
    },
    "usage": { "input_tokens": 1104, "output_tokens": 102 }
  },
  "free helpdesk software": {
    "model": "jev-1.14.0",
    "answers": {
      "intent": {
        "type": "choice",
        "choice": "Commercial",
        "confidence": 0.62,
        "probabilities": { "Informational": 0.08, "Navigational": 0.01, "Commercial": 0.74, "Transactional": 0.17 }
      },
      "enterprise": { "type": "noul", "noul": 0.21 },
      "direct_fit": { "type": "noul", "noul": 0.91 },
      "urgency":    { "type": "noul", "noul": 0.66 }
    },
    "usage": { "input_tokens": 1098, "output_tokens": 102 }
  },
  "crm for real estate agents": {
    "model": "jev-1.14.0",
    "answers": {
      "intent": {
        "type": "choice",
        "choice": "Commercial",
        "confidence": 0.80,
        "probabilities": { "Informational": 0.07, "Navigational": 0.0, "Commercial": 0.88, "Transactional": 0.05 }
      },
      "enterprise": { "type": "noul", "noul": 0.34 },
      "direct_fit": { "type": "noul", "noul": 0.11 },
      "urgency":    { "type": "noul", "noul": 0.61 }
    },
    "usage": { "input_tokens": 1101, "output_tokens": 102 }
  }
}

Look at “free helpdesk software”: a fit (0.91), but almost certainly not an enterprise buyer (0.21). That’s exactly the call a person would make by hand, now made the same way on all 2,000 rows.


Step 4: Pick the keywords (the rules live in code)

Now code takes over, in two stages: hard filters (pass or fail) and ranking (best first).

The hard filters

A keyword must pass all of these:

  1. Intent is Commercial or Transactional. Navigational only counts for the advertiser’s own brand.
  2. Direct fit ≥ 0.5. HelpNest actually does this.
  3. Enterprise ≥ 0.5. A business buyer, not a hobbyist.
  4. Searches > 0.
  5. 0 < CPC < $30.
reasons = []
if not (intent in ('Transactional', 'Commercial') or (intent == 'Navigational' and own_brand)):
    reasons.append(f'intent {intent}')
if p['direct_fit'] < YES:
    reasons.append('not a direct fit')
if p['enterprise'] < YES:
    reasons.append('not enterprise')
if searches <= 0:
    reasons.append('zero searches')
if cpc <= 0:
    reasons.append('no bid data (CPC 0)')
elif cpc >= CPC_MAX:
    reasons.append(f'CPC ${cpc:g} >= ${CPC_MAX:g}')

Every failure records a reason. The output CSV has a “Not picked because” column, so nobody has to guess.

The ranking

Everything that passes gets a score:

signal = p['direct_fit'] * (p['enterprise'] + p['urgency']) / 2
boost  = 1 + BOOST * math.log10(searches / 50)   # BOOST = 0.10
score  = signal * boost

In plain English:

  • Signal score = how well it fits × the average of “how enterprise” and “how ready to buy”.
  • Volume boost = a small bonus for more searches: 50 searches → ×1.0, 500 → ×1.1, 5,000 → ×1.2. It’s deliberately small, so volume breaks ties between similar keywords but can’t rescue a weak one.

Sort by score, take the top 100. Done.

Four lessons from the data

The first draft of the rules looked sensible and was wrong in four ways. Checking the data caught each one:

  • CPC = 0 doesn’t mean cheap. It means “no bid data.” Over half the list had CPC 0. A rule saying “CPC under $30” would have let all of them in. Fix: 0 < CPC < $30.
  • “Navigational” is a trap. Navigational means “looking for a specific website”, and in a keyword list that’s mostly competitors’ brand names (“deskpilot login”). Fix: allow Navigational only for the advertiser’s own brand, checked by an exact text match in code.
  • Too many must-haves leave nothing. Requiring Commercial and enterprise and fit and urgency and CPC left only a handful of keywords. People rarely type urgency into a search box. Fix: urgency became a ranking signal, not a filter.
  • Test each filter on its own. “Enterprise” was briefly demoted to a ranking signal too, and some small-business keywords crept in. On a closer look, enterprise wasn’t the filter removing everything; intent was. Enterprise went back to being a hard filter. Check what each condition removes before loosening it.

None of these are AI problems. They’re the same lessons Excel teaches, except here changing a rule takes one line and a one-second re-run.


Step 5: Sort into ad groups

Same pattern, different question. With an LLM’s help, 12 ad groups were drafted, each written so that one ad and one landing page could answer every keyword in the group:

Competitor alternatives · Helpdesk software · Ticketing systems · Live chat & messaging · Shared inbox · Knowledge base · AI support & chatbots · Customer service automation · Support analytics & SLAs · IT service desk · E-commerce support · Industry support (healthcare, finance, education)

The rule for keywords that fit two groups: most specific wins. A named competitor comes first, then a named industry or channel, then the general solution type.

Jev gets one Choice question (“which group?”) plus a safety-net Noul question (“does this clearly belong in any group?”). If the second answer is below 0.5, the row is marked CHECK instead of being forced into a bad fit.

jev-adgroups.json (3 records)

{
  "deskpilot alternatives": {
    "model": "jev-1.14.0",
    "answers": {
      "ad_group": {
        "type": "choice",
        "choice": "Competitor alternatives",
        "confidence": 1.0,
        "probabilities": { "Competitor alternatives": 1.0, "Helpdesk software": 0.0, "Ticketing systems": 0.0, "...": 0.0 }
      },
      "ad_group_any": { "type": "noul", "noul": 0.97 }
    },
    "usage": { "input_tokens": 1762, "output_tokens": 160 }
  },
  "omnichannel ticketing software": {
    "model": "jev-1.14.0",
    "answers": {
      "ad_group": {
        "type": "choice",
        "choice": "Ticketing systems",
        "confidence": 0.81,
        "probabilities": { "Ticketing systems": 0.88, "Live chat & messaging": 0.07, "Helpdesk software": 0.04, "...": 0.01 }
      },
      "ad_group_any": { "type": "noul", "noul": 0.95 }
    },
    "usage": { "input_tokens": 1765, "output_tokens": 160 }
  },
  "support desk for clinics": {
    "model": "jev-1.14.0",
    "answers": {
      "ad_group": {
        "type": "choice",
        "choice": "Industry support (healthcare, finance, education)",
        "confidence": 0.94,
        "probabilities": { "Industry support (healthcare, finance, education)": 0.96, "Helpdesk software": 0.03, "...": 0.01 }
      },
      "ad_group_any": { "type": "noul", "noul": 0.91 }
    },
    "usage": { "input_tokens": 1764, "output_tokens": 160 }
  }
}

(Probabilities trimmed with “…” for readability. The real file lists all 12 groups.)


The result: picked-keywords.csv

One Excel-friendly file: the original columns, Jev’s answers with their probabilities, the scores, the ad group, and the reason a keyword wasn’t picked. The full file has 22 columns; here are the ones that matter most, for the top 5 of the 100:

Rank Keyword Ad group Searches CPC Enterprise Fit Urgency Pick score
1 help desk ticketing system for enterprise Ticketing systems 500 $25 0.88 0.97 0.70 0.843
2 omnichannel ticketing software Ticketing systems 500 $21 0.83 0.95 0.66 0.779
3 deskpilot alternatives Competitor alternatives 50 $14 0.82 0.96 0.80 0.778
4 customer service automation software Customer service automation 5,000 $19 0.79 0.93 0.58 0.764
5 supporthive pricing Competitor alternatives 50 $11 0.71 0.95 0.88 0.755

CPC is the geometric midpoint of the top-of-page bid range. The full file also includes intent and its confidence, YES/NO columns, the signal score and volume boost separately, ad-group confidence, and the “Not picked because” column.

Look at #2 and #3: “deskpilot alternatives” has the slightly stronger signal (0.778 vs 0.708), but “omnichannel ticketing software” gets 10× the searches, and the small volume boost nudges it ahead. That’s the tie-breaker doing its job without overpowering quality.

The rows that didn’t make it explain themselves:

Keyword Not picked because
free helpdesk software not enterprise
what is a ticketing system intent Informational
crm for real estate agents not a direct fit; not enterprise
deskpilot login intent Navigational; no bid data (CPC 0)
some keyword ranked #131 qualified, ranked 131 (top 100 picked)

Limits, and when to check by hand

This workflow removes the scroll, not the marketer. Keep an eye on these:

  • Close calls are real. Plenty of keywords land between 0.4 and 0.6 on enterprise or fit. A 0.49 and a 0.51 are nearly the same judgment, yet one passes and one doesn’t. Sort the output by probability and skim anything near the cut.
  • A tidy answer isn’t automatically a right answer. Jev always returns a valid label, but the label can still be wrong. Claims that a model like this “can’t hallucinate” only mean the answer always fits the format, not that it’s correct. Before running all 2,000, run 20–25 keywords spread across the list and read every answer. If something looks off, the fix is almost always a clearer definition in the question or the company description.
  • Low confidence and CHECK rows deserve a look. An ad group chosen at 0.5 confidence often means the keyword sits between two groups. Decide those by hand using the “most specific wins” rule.
  • Near-duplicates eat slots. “deskpilot alternative” and “deskpilot alternatives” can both rank highly. Google Ads matches close variants anyway, so keep one and free the slot.
  • Some keywords are strategic, not scored. A new category term you want to own may score low today because searchers are still learning about it. Add those by hand as their own small ad group so their results stay separate.
  • The picks are a starting point. Campaign data (clicks, conversions, cost per lead) beats any pre-launch score. After two weeks, use real results to adjust thresholds and weights, then re-pick.
  • The probabilities are for ranking, not promises. “Enterprise: 0.82” is good for sorting and setting a cut-off. It doesn’t mean 82% of those searchers are enterprise buyers. Before trusting a cut-off like 0.5, label 50 keywords yourself and check that the high scores really are the better keywords.
  • Your data goes to an API. Keyword lists are usually fine to send. A client’s private data, CRM exports or unreleased product names may not be. Self-hosted alternatives exist, but they need your own GPU and some setup work.

It doesn’t have to be Jev

Jev isn’t the only decision model. Others have appeared since, including OpenAI’s decisions API and open-weight models you can run locally. As one commenter put it, a lot of what people have been using LLMs for would be better suited to a classifier.

The workflow doesn’t depend on the vendor: judgments from a model, rules in code, answers saved to a file. Any model that returns probabilities can fill Jev’s slot, so swapping later means changing the ask() function, not the rules.


What this changes in a marketer’s week

  • From hours to about a minute. ~2,000 keywords, 8,000 judgment calls, about 65 seconds, under 10 cents.
  • From gut feel to written rules. The criteria live in the questions and the code, not in someone’s head.
  • From “why is this here?” to a reason on every row.
  • From “redo everything” to “change one number.” New CPC cap? Want 150 keywords? Re-run in a second, with no new AI calls.
  • The marketer still decides. You define what “enterprise” means, which filters matter, and how many keywords to pick. Jev applies those definitions consistently, thousands of times.

The best part isn’t the speed. When the first two weeks of campaign data come in, a threshold or weight can be adjusted and the list re-picked in seconds, with every change written down.


Want to try it? You need a keyword export, a TypeSafe API key, and about 300 lines of Python. Start with 20 keywords (--sample 20) and read the answers before running all 2,000. If the answers look off, the fix is almost always a clearer definition in the questions.