fix(journaling): enforce stricter LLM evaluation rules for YouTube video selection
All checks were successful
Monorepo Pipeline / ⚡ Prioritize Release (push) Successful in 2s
Monorepo Pipeline / 🧪 Test (push) Successful in 2m53s
Monorepo Pipeline / 🏗️ Build (push) Successful in 2m29s
Monorepo Pipeline / 🧹 Lint (push) Successful in 3m37s
Monorepo Pipeline / 🚀 Release (push) Has been skipped
Monorepo Pipeline / 🐳 Build Directus (Base) (push) Has been skipped
Monorepo Pipeline / 🐳 Build Build-Base (push) Has been skipped
Monorepo Pipeline / 🐳 Build Gatekeeper (Product) (push) Has been skipped
Monorepo Pipeline / 🐳 Build Production Runtime (push) Has been skipped
All checks were successful
Monorepo Pipeline / ⚡ Prioritize Release (push) Successful in 2s
Monorepo Pipeline / 🧪 Test (push) Successful in 2m53s
Monorepo Pipeline / 🏗️ Build (push) Successful in 2m29s
Monorepo Pipeline / 🧹 Lint (push) Successful in 3m37s
Monorepo Pipeline / 🚀 Release (push) Has been skipped
Monorepo Pipeline / 🐳 Build Directus (Base) (push) Has been skipped
Monorepo Pipeline / 🐳 Build Build-Base (push) Has been skipped
Monorepo Pipeline / 🐳 Build Gatekeeper (Product) (push) Has been skipped
Monorepo Pipeline / 🐳 Build Production Runtime (push) Has been skipped
This commit is contained in:
@@ -195,11 +195,11 @@ Return JSON: { "facts": [ { "statement": "...", "source": "Organization Name Onl
|
||||
RULES:
|
||||
1. Extract only the 2-4 most important technical or business keywords from the provided text.
|
||||
2. Ignore all markdown syntax, frontmatter (---), titles, and descriptions.
|
||||
3. Keep the query generic enough to find popular educational tech videos.
|
||||
3. Keep the query generic enough to find popular educational tech videos, BUT ensure it specifically targets the core technical subject. Append "tutorial" or "b2b explanation" if necessary to find high-quality content.
|
||||
4. DO NOT append specific channel names (e.g., "Fireship", "Vercel") to the query.
|
||||
5. DO NOT USE QUOTES IN THE QUERY.
|
||||
|
||||
Return a JSON object with a single string field "query". Example: {"query": "core web vitals performance"}`,
|
||||
Return a JSON object with a single string field "query". Example: {"query": "core web vitals performance tutorial"}`,
|
||||
},
|
||||
{
|
||||
role: "user",
|
||||
@@ -243,15 +243,15 @@ Return a JSON object with a single string field "query". Example: {"query": "cor
|
||||
}
|
||||
|
||||
// Step 3: Ask the LLM to evaluate the relevance of the found videos
|
||||
const evalPrompt = `You are a strict technical evaluator. You must select the MOST RELEVANT educational tech video from the list below based on this context: "${topic.slice(0, 500)}..."
|
||||
const evalPrompt = `You are a strict technical evaluator. You must select the MOST RELEVANT educational tech video from the list below based on this core article context: "${topic.slice(0, 800)}..."
|
||||
|
||||
Videos:
|
||||
${ytVideos.map((v, i) => `[ID: ${i}] Title: "${v.title}" | Channel: "${v.channel}" | Snippet: "${v.snippet || "none"}"`).join("\n")}
|
||||
${ytVideos.map((v, i) => `[ID: ${i}] Title: "${v.title}" | Channel: "${v.channel}" | Snippet: "${v.snippet || 'none'}"`).join("\n")}
|
||||
|
||||
RULES:
|
||||
1. The video MUST be highly relevant to the context.
|
||||
2. The channel SHOULD be a high-quality tech, development, or professional B2B channel (e.g., Google Developers, Vercel, Theo - t3.gg, Fireship, Syntax, ByteByteGo, IBM Technology, McKinsey, Gartner, Deloitte). AVOID gaming, generic vlogs, clickbait, or unrelated topics.
|
||||
3. If none are truly relevant, return -1.
|
||||
1. The video MUST be highly relevant to the EXACT technical topic of the context.
|
||||
2. The channel SHOULD be a high-quality tech, development, or professional B2B channel (e.g., Google Developers, Vercel, Theo - t3.gg, Fireship, Syntax, ByteByteGo, IBM Technology, McKinsey, Gartner, Deloitte). AVOID gaming, generic vlogs, clickbait, off-topic podcasts, or unrelated topics.
|
||||
3. If none of the videos are strictly relevant to the core technical or business subject (e.g. they are just casually mentioning the word), YOU MUST RETURN -1. Be extremely critical. Do not just pick the "best of the worst".
|
||||
4. If one is highly relevant, return its ID number.
|
||||
|
||||
Return ONLY a JSON object: {"bestVideoId": number}`;
|
||||
@@ -343,7 +343,7 @@ CRITICAL: Do NOT provide more than 2 trendsKeywords. Keep it extremely focused.`
|
||||
try {
|
||||
let parsed = JSON.parse(
|
||||
response.choices[0].message.content ||
|
||||
'{"trendsKeywords": [], "dcVariables": []}',
|
||||
'{"trendsKeywords": [], "dcVariables": []}',
|
||||
);
|
||||
if (Array.isArray(parsed)) {
|
||||
parsed = parsed[0] || { trendsKeywords: [], dcVariables: [] };
|
||||
|
||||
Reference in New Issue
Block a user