[{"data":1,"prerenderedAt":957},["ShallowReactive",2],{"article-alternates":3,"article-\u002Fes\u002Fai\u002Fversionamiento-de-prompts-y-pruebas-ab-disciplina-en-operaciones-llm":13},{"i18nKey":4,"paths":5},"ai-004-2026-06",{"de":6,"en":7,"es":8,"fr":9,"it":10,"ru":11,"tr":12},"\u002Fde\u002Fai\u002Fprompt-versionierung-und-ab-test-llm-operations-disziplin","\u002Fen\u002Fai\u002Fprompt-versioning-and-ab-testing-llm-operations-discipline","\u002Fes\u002Fai\u002Fversionamiento-de-prompts-y-pruebas-ab-disciplina-en-operaciones-llm","\u002Ffr\u002Fai\u002Fversionnage-des-prompts-et-tests-ab-discipline-des-operations-llm","\u002Fit\u002Fai\u002Fversionamento-prompt-e-test-ab-discipline-operativa-llm","\u002Fru\u002Fai\u002Fversiyonlama-t","\u002Ftr\u002Fai\u002Fprompt-versiyonlama-ve-a-b-testi-llm-operasyonun-disiplini",{"_path":8,"_dir":14,"_draft":15,"_partial":15,"_locale":16,"title":17,"description":18,"publishedAt":19,"modifiedAt":19,"category":14,"i18nKey":4,"tags":20,"readingTime":26,"author":27,"body":28,"_type":951,"_id":952,"_source":953,"_file":954,"_stem":955,"_extension":956},"ai",false,"","Versionamiento de Prompts y Pruebas A\u002FB: Disciplina en Operaciones LLM","Con Promptfoo, LangSmith y pipelines de evaluación, convierte cambios de prompts en medibles. Cómo configurar versionamiento y A\u002FB testing en LLM production.","2026-06-22",[21,22,23,24,25],"prompt-engineering","llm-ops","evaluación","ab-testing","promptfoo",8,"Roibase",{"type":29,"children":30,"toc":942},"root",[31,39,46,51,56,61,67,72,293,298,303,316,412,417,423,428,457,495,500,586,591,597,602,607,754,759,764,770,775,788,854,867,881,887,892,897,905,910,915,921,926,931,936],{"type":32,"tag":33,"props":34,"children":35},"element","p",{},[36],{"type":37,"value":38},"text","Ejecutar LLMs en production ya no se trata solo de llamadas a API. Cuando cambias un prompt, la calidad del output puede caer un 15% o subir un 22% — pero si no lo detectas, el deployment se convierte en aleatoriedad. Versionamiento de prompts y A\u002FB testing traen la disciplina de deployment de software a las operaciones LLM. Este artículo explica cómo usar frameworks de evaluación como Promptfoo y LangSmith para hacer medibles los cambios de prompts.",{"type":32,"tag":40,"props":41,"children":43},"h2",{"id":42},"cambiar-un-prompt-no-es-un-deployment",[44],{"type":37,"value":45},"Cambiar un prompt no es un deployment",{"type":32,"tag":33,"props":47,"children":48},{},[49],{"type":37,"value":50},"En ingeniería de software clásica, cuando una función cambia, entran en juego unit tests, integration tests y canary deployments. En operaciones LLM, la mayoría de equipos cambian el prompt en un archivo de texto plano, hacen algunas pruebas manuales y lo envían a production. El resultado: el sentiment del usuario cae un 8%, pero nadie lo conecta.",{"type":32,"tag":33,"props":52,"children":53},{},[54],{"type":37,"value":55},"El problema es este: el output de LLM no es determinístico. Obtienes respuestas diferentes para el mismo prompt, lo que hace que probar con un solo ejemplo sea sin sentido. Sin un sistema de versionamiento, no puedes responder \"¿era mejor el prompt antiguo o el nuevo?\" Git commit tampoco es suficiente — no puedes extraer la diferencia semántica del message del commit.",{"type":32,"tag":33,"props":57,"children":58},{},[59],{"type":37,"value":60},"La solución: registra cada cambio de prompt como una versión, ejecuta tu evaluation set antes y después del cambio, compara métricas. Esta disciplina logra dos cosas: detección de regresión (si el nuevo prompt rompe tareas antiguas) y medición de mejora (si la métrica que apuntabas realmente subió).",{"type":32,"tag":40,"props":62,"children":64},{"id":63},"cómo-montar-un-pipeline-de-evaluación",[65],{"type":37,"value":66},"Cómo montar un pipeline de evaluación",{"type":32,"tag":33,"props":68,"children":69},{},[70],{"type":37,"value":71},"Un pipeline de evaluación tiene tres componentes: eval set, métrica de evaluación y runner. El eval set es una lista de inputs que enviarás al LLM y los outputs esperados (o propiedades del output). Se ve así en JSON:",{"type":32,"tag":73,"props":74,"children":78},"pre",{"code":75,"language":76,"meta":16,"className":77,"style":16},"[\n  {\n    \"input\": \"Resume la tendencia de ingresos Q1 2025\",\n    \"expected_topics\": [\"ingresos\", \"crecimiento\", \"trimestre\"],\n    \"expected_sentiment\": \"neutral\"\n  },\n  {\n    \"input\": \"Explica por qué la tasa de churn subió\",\n    \"expected_topics\": [\"churn\", \"retención\"],\n    \"expected_sentiment\": \"analítico\"\n  }\n]\n","json","language-json shiki shiki-themes github-dark",[79],{"type":32,"tag":80,"props":81,"children":82},"code",{"__ignoreMap":16},[83,95,104,130,173,191,200,208,228,258,275,284],{"type":32,"tag":84,"props":85,"children":88},"span",{"class":86,"line":87},"line",1,[89],{"type":32,"tag":84,"props":90,"children":92},{"style":91},"--shiki-default:#E1E4E8",[93],{"type":37,"value":94},"[\n",{"type":32,"tag":84,"props":96,"children":98},{"class":86,"line":97},2,[99],{"type":32,"tag":84,"props":100,"children":101},{"style":91},[102],{"type":37,"value":103},"  {\n",{"type":32,"tag":84,"props":105,"children":107},{"class":86,"line":106},3,[108,114,119,125],{"type":32,"tag":84,"props":109,"children":111},{"style":110},"--shiki-default:#79B8FF",[112],{"type":37,"value":113},"    \"input\"",{"type":32,"tag":84,"props":115,"children":116},{"style":91},[117],{"type":37,"value":118},": ",{"type":32,"tag":84,"props":120,"children":122},{"style":121},"--shiki-default:#9ECBFF",[123],{"type":37,"value":124},"\"Resume la tendencia de ingresos Q1 2025\"",{"type":32,"tag":84,"props":126,"children":127},{"style":91},[128],{"type":37,"value":129},",\n",{"type":32,"tag":84,"props":131,"children":133},{"class":86,"line":132},4,[134,139,144,149,154,159,163,168],{"type":32,"tag":84,"props":135,"children":136},{"style":110},[137],{"type":37,"value":138},"    \"expected_topics\"",{"type":32,"tag":84,"props":140,"children":141},{"style":91},[142],{"type":37,"value":143},": [",{"type":32,"tag":84,"props":145,"children":146},{"style":121},[147],{"type":37,"value":148},"\"ingresos\"",{"type":32,"tag":84,"props":150,"children":151},{"style":91},[152],{"type":37,"value":153},", ",{"type":32,"tag":84,"props":155,"children":156},{"style":121},[157],{"type":37,"value":158},"\"crecimiento\"",{"type":32,"tag":84,"props":160,"children":161},{"style":91},[162],{"type":37,"value":153},{"type":32,"tag":84,"props":164,"children":165},{"style":121},[166],{"type":37,"value":167},"\"trimestre\"",{"type":32,"tag":84,"props":169,"children":170},{"style":91},[171],{"type":37,"value":172},"],\n",{"type":32,"tag":84,"props":174,"children":176},{"class":86,"line":175},5,[177,182,186],{"type":32,"tag":84,"props":178,"children":179},{"style":110},[180],{"type":37,"value":181},"    \"expected_sentiment\"",{"type":32,"tag":84,"props":183,"children":184},{"style":91},[185],{"type":37,"value":118},{"type":32,"tag":84,"props":187,"children":188},{"style":121},[189],{"type":37,"value":190},"\"neutral\"\n",{"type":32,"tag":84,"props":192,"children":194},{"class":86,"line":193},6,[195],{"type":32,"tag":84,"props":196,"children":197},{"style":91},[198],{"type":37,"value":199},"  },\n",{"type":32,"tag":84,"props":201,"children":203},{"class":86,"line":202},7,[204],{"type":32,"tag":84,"props":205,"children":206},{"style":91},[207],{"type":37,"value":103},{"type":32,"tag":84,"props":209,"children":210},{"class":86,"line":26},[211,215,219,224],{"type":32,"tag":84,"props":212,"children":213},{"style":110},[214],{"type":37,"value":113},{"type":32,"tag":84,"props":216,"children":217},{"style":91},[218],{"type":37,"value":118},{"type":32,"tag":84,"props":220,"children":221},{"style":121},[222],{"type":37,"value":223},"\"Explica por qué la tasa de churn subió\"",{"type":32,"tag":84,"props":225,"children":226},{"style":91},[227],{"type":37,"value":129},{"type":32,"tag":84,"props":229,"children":231},{"class":86,"line":230},9,[232,236,240,245,249,254],{"type":32,"tag":84,"props":233,"children":234},{"style":110},[235],{"type":37,"value":138},{"type":32,"tag":84,"props":237,"children":238},{"style":91},[239],{"type":37,"value":143},{"type":32,"tag":84,"props":241,"children":242},{"style":121},[243],{"type":37,"value":244},"\"churn\"",{"type":32,"tag":84,"props":246,"children":247},{"style":91},[248],{"type":37,"value":153},{"type":32,"tag":84,"props":250,"children":251},{"style":121},[252],{"type":37,"value":253},"\"retención\"",{"type":32,"tag":84,"props":255,"children":256},{"style":91},[257],{"type":37,"value":172},{"type":32,"tag":84,"props":259,"children":261},{"class":86,"line":260},10,[262,266,270],{"type":32,"tag":84,"props":263,"children":264},{"style":110},[265],{"type":37,"value":181},{"type":32,"tag":84,"props":267,"children":268},{"style":91},[269],{"type":37,"value":118},{"type":32,"tag":84,"props":271,"children":272},{"style":121},[273],{"type":37,"value":274},"\"analítico\"\n",{"type":32,"tag":84,"props":276,"children":278},{"class":86,"line":277},11,[279],{"type":32,"tag":84,"props":280,"children":281},{"style":91},[282],{"type":37,"value":283},"  }\n",{"type":32,"tag":84,"props":285,"children":287},{"class":86,"line":286},12,[288],{"type":32,"tag":84,"props":289,"children":290},{"style":91},[291],{"type":37,"value":292},"]\n",{"type":32,"tag":33,"props":294,"children":295},{},[296],{"type":37,"value":297},"Puedes crear el eval set manualmente (muestreando de tus logs de production) o generarlo sintéticamente (preguntándole a otro LLM \"genera 50 variaciones de queries para este prompt\"). Lo importante es que el set cubra edge cases — inputs largos, queries ambiguas, múltiples idiomas.",{"type":32,"tag":33,"props":299,"children":300},{},[301],{"type":37,"value":302},"La métrica de evaluación define cómo puntuarás el output del LLM. Hay dos tipos comunes: basada en reglas (verificar la presencia de palabras específicas en el output) y LLM-as-judge (preguntarle a otro LLM \"¿contesta correctamente esta pregunta? Puntúa del 1 al 5\"). LLM-as-judge es más flexible pero más costoso y lento. Para un balance entre velocidad y precisión, la combinación de reglas + clasificador lightweight (como un modelo de sentimientos basado en BERT) es preferible.",{"type":32,"tag":33,"props":304,"children":305},{},[306,308,314],{"type":37,"value":307},"El runner toma el eval set, ejecuta tanto el prompt antiguo como el nuevo para cada input, compara los outputs con la métrica y genera una tabla de diferencias. Promptfoo lo hace desde la terminal con ",{"type":32,"tag":80,"props":309,"children":311},{"className":310},[],[312],{"type":37,"value":313},"promptfoo eval",{"type":37,"value":315},":",{"type":32,"tag":73,"props":317,"children":321},{"code":318,"language":319,"meta":16,"className":320,"style":16},"promptfoo eval \\\n  --prompts prompts\u002Fv1.txt prompts\u002Fv2.txt \\\n  --providers openai:gpt-4 \\\n  --tests evals\u002Fsummarization.json \\\n  --output results.json\n","bash","language-bash shiki shiki-themes github-dark",[322],{"type":32,"tag":80,"props":323,"children":324},{"__ignoreMap":16},[325,343,365,382,399],{"type":32,"tag":84,"props":326,"children":327},{"class":86,"line":87},[328,333,338],{"type":32,"tag":84,"props":329,"children":331},{"style":330},"--shiki-default:#B392F0",[332],{"type":37,"value":25},{"type":32,"tag":84,"props":334,"children":335},{"style":121},[336],{"type":37,"value":337}," eval",{"type":32,"tag":84,"props":339,"children":340},{"style":110},[341],{"type":37,"value":342}," \\\n",{"type":32,"tag":84,"props":344,"children":345},{"class":86,"line":97},[346,351,356,361],{"type":32,"tag":84,"props":347,"children":348},{"style":110},[349],{"type":37,"value":350},"  --prompts",{"type":32,"tag":84,"props":352,"children":353},{"style":121},[354],{"type":37,"value":355}," prompts\u002Fv1.txt",{"type":32,"tag":84,"props":357,"children":358},{"style":121},[359],{"type":37,"value":360}," prompts\u002Fv2.txt",{"type":32,"tag":84,"props":362,"children":363},{"style":110},[364],{"type":37,"value":342},{"type":32,"tag":84,"props":366,"children":367},{"class":86,"line":106},[368,373,378],{"type":32,"tag":84,"props":369,"children":370},{"style":110},[371],{"type":37,"value":372},"  --providers",{"type":32,"tag":84,"props":374,"children":375},{"style":121},[376],{"type":37,"value":377}," openai:gpt-4",{"type":32,"tag":84,"props":379,"children":380},{"style":110},[381],{"type":37,"value":342},{"type":32,"tag":84,"props":383,"children":384},{"class":86,"line":132},[385,390,395],{"type":32,"tag":84,"props":386,"children":387},{"style":110},[388],{"type":37,"value":389},"  --tests",{"type":32,"tag":84,"props":391,"children":392},{"style":121},[393],{"type":37,"value":394}," evals\u002Fsummarization.json",{"type":32,"tag":84,"props":396,"children":397},{"style":110},[398],{"type":37,"value":342},{"type":32,"tag":84,"props":400,"children":401},{"class":86,"line":175},[402,407],{"type":32,"tag":84,"props":403,"children":404},{"style":110},[405],{"type":37,"value":406},"  --output",{"type":32,"tag":84,"props":408,"children":409},{"style":121},[410],{"type":37,"value":411}," results.json\n",{"type":32,"tag":33,"props":413,"children":414},{},[415],{"type":37,"value":416},"En el output verás cuál prompt tiene mejor desempeño para cada caso de prueba. Si el nuevo prompt mejora la métrica en el 80% del eval set, está listo para deployment. Si no, hay regresión — revisa el prompt.",{"type":32,"tag":40,"props":418,"children":420},{"id":419},"ab-testing-ejecutar-dos-prompts-en-paralelo-en-production",[421],{"type":37,"value":422},"A\u002FB testing: ejecutar dos prompts en paralelo en production",{"type":32,"tag":33,"props":424,"children":425},{},[426],{"type":37,"value":427},"El pipeline de evaluación da resultados offline — sin datos de usuarios reales. Para ejecutar dos prompts simultáneamente en production y medir cuál funciona mejor, necesitas A\u002FB testing. Para esto se requiere infraestructura de traffic splitting y recolección de métricas.",{"type":32,"tag":33,"props":429,"children":430},{},[431,433,439,441,447,449,455],{"type":37,"value":432},"El traffic splitting es simple: tomas el request entrante, hashea su ",{"type":32,"tag":80,"props":434,"children":436},{"className":435},[],[437],{"type":37,"value":438},"user_id",{"type":37,"value":440}," o ",{"type":32,"tag":80,"props":442,"children":444},{"className":443},[],[445],{"type":37,"value":446},"session_id",{"type":37,"value":448},", aplicas modulo y basado en el resultado diriges a prompt A o B. Por ejemplo, si ",{"type":32,"tag":80,"props":450,"children":452},{"className":451},[],[453],{"type":37,"value":454},"hash(user_id) % 100 \u003C 50",{"type":37,"value":456},", envías a A; si no, a B. Así logras un split 50-50. Lo importante: el mismo usuario debe ver el mismo prompt en cada request (asignación sticky) — de lo contrario la experiencia es inconsistente.",{"type":32,"tag":33,"props":458,"children":459},{},[460,462,468,469,475,476,482,484,493],{"type":37,"value":461},"Para recolección de métricas, añades metadata junto a la respuesta del LLM: ",{"type":32,"tag":80,"props":463,"children":465},{"className":464},[],[466],{"type":37,"value":467},"prompt_version",{"type":37,"value":153},{"type":32,"tag":80,"props":470,"children":472},{"className":471},[],[473],{"type":37,"value":474},"latency",{"type":37,"value":153},{"type":32,"tag":80,"props":477,"children":479},{"className":478},[],[480],{"type":37,"value":481},"token_count",{"type":37,"value":483},". Estos datos fluyen a tu data warehouse (BigQuery, Snowflake). La infraestructura de ",{"type":32,"tag":485,"props":486,"children":490},"a",{"href":487,"rel":488},"https:\u002F\u002Fwww.roibase.com.tr\u002Fes\u002Fverianalizi",[489],"nofollow",[491],{"type":37,"value":492},"Veri Analizi & İçgörü Mühendisliği",{"type":37,"value":494}," de Roibase entra aquí — combinas logs de LLM con otros eventos (acciones del usuario, conversión, churn) para medir el impacto downstream del prompt.",{"type":32,"tag":33,"props":496,"children":497},{},[498],{"type":37,"value":499},"¿Qué métricas monitoreas en A\u002FB testing? Tres categorías:",{"type":32,"tag":501,"props":502,"children":503},"table",{},[504,528],{"type":32,"tag":505,"props":506,"children":507},"thead",{},[508],{"type":32,"tag":509,"props":510,"children":511},"tr",{},[512,518,523],{"type":32,"tag":513,"props":514,"children":515},"th",{},[516],{"type":37,"value":517},"Tipo de métrica",{"type":32,"tag":513,"props":519,"children":520},{},[521],{"type":37,"value":522},"Ejemplo",{"type":32,"tag":513,"props":524,"children":525},{},[526],{"type":37,"value":527},"Objetivo",{"type":32,"tag":529,"props":530,"children":531},"tbody",{},[532,551,569],{"type":32,"tag":509,"props":533,"children":534},{},[535,541,546],{"type":32,"tag":536,"props":537,"children":538},"td",{},[539],{"type":37,"value":540},"Calidad",{"type":32,"tag":536,"props":542,"children":543},{},[544],{"type":37,"value":545},"Puntuación LLM-as-judge, tasa de alucinación",{"type":32,"tag":536,"props":547,"children":548},{},[549],{"type":37,"value":550},"Alto",{"type":32,"tag":509,"props":552,"children":553},{},[554,559,564],{"type":32,"tag":536,"props":555,"children":556},{},[557],{"type":37,"value":558},"Costo",{"type":32,"tag":536,"props":560,"children":561},{},[562],{"type":37,"value":563},"Token count, costo de API",{"type":32,"tag":536,"props":565,"children":566},{},[567],{"type":37,"value":568},"Bajo",{"type":32,"tag":509,"props":570,"children":571},{},[572,577,582],{"type":32,"tag":536,"props":573,"children":574},{},[575],{"type":37,"value":576},"Downstream",{"type":32,"tag":536,"props":578,"children":579},{},[580],{"type":37,"value":581},"Tasa de conversión, engagement del usuario",{"type":32,"tag":536,"props":583,"children":584},{},[585],{"type":37,"value":550},{"type":32,"tag":33,"props":587,"children":588},{},[589],{"type":37,"value":590},"Por ejemplo, si el prompt B sube la puntuación LLM-as-judge un 12% vs A pero aumenta el token count un 35%, hay un tradeoff. Si la conversión downstream no cambia, el prompt A es más eficiente.",{"type":32,"tag":40,"props":592,"children":594},{"id":593},"langsmith-y-observability",[595],{"type":37,"value":596},"LangSmith y observability",{"type":32,"tag":33,"props":598,"children":599},{},[600],{"type":37,"value":601},"LangSmith es una plataforma de observabilidad LLM desarrollada por el equipo de LangChain. Más allá de evaluación, captura traces de production, visualiza cadenas de prompts y muestra dónde se incrementa la latencia. Es especialmente crítico en workflows LLM multi-paso (RAG + summarización + JSON parsing).",{"type":32,"tag":33,"props":603,"children":604},{},[605],{"type":37,"value":606},"Enviar traces a LangSmith usa el SDK:",{"type":32,"tag":73,"props":608,"children":612},{"code":609,"language":610,"meta":16,"className":611,"style":16},"from langsmith import Client\nclient = Client(api_key=\"...\")\n\nwith client.trace(name=\"summarize_revenue\"):\n    result = llm.invoke(prompt)\n    client.log_metric(\"token_count\", result.usage.total_tokens)\n","python","language-python shiki shiki-themes github-dark",[613],{"type":32,"tag":80,"props":614,"children":615},{"__ignoreMap":16},[616,640,678,687,719,736],{"type":32,"tag":84,"props":617,"children":618},{"class":86,"line":87},[619,625,630,635],{"type":32,"tag":84,"props":620,"children":622},{"style":621},"--shiki-default:#F97583",[623],{"type":37,"value":624},"from",{"type":32,"tag":84,"props":626,"children":627},{"style":91},[628],{"type":37,"value":629}," langsmith ",{"type":32,"tag":84,"props":631,"children":632},{"style":621},[633],{"type":37,"value":634},"import",{"type":32,"tag":84,"props":636,"children":637},{"style":91},[638],{"type":37,"value":639}," Client\n",{"type":32,"tag":84,"props":641,"children":642},{"class":86,"line":97},[643,648,653,658,664,668,673],{"type":32,"tag":84,"props":644,"children":645},{"style":91},[646],{"type":37,"value":647},"client ",{"type":32,"tag":84,"props":649,"children":650},{"style":621},[651],{"type":37,"value":652},"=",{"type":32,"tag":84,"props":654,"children":655},{"style":91},[656],{"type":37,"value":657}," Client(",{"type":32,"tag":84,"props":659,"children":661},{"style":660},"--shiki-default:#FFAB70",[662],{"type":37,"value":663},"api_key",{"type":32,"tag":84,"props":665,"children":666},{"style":621},[667],{"type":37,"value":652},{"type":32,"tag":84,"props":669,"children":670},{"style":121},[671],{"type":37,"value":672},"\"...\"",{"type":32,"tag":84,"props":674,"children":675},{"style":91},[676],{"type":37,"value":677},")\n",{"type":32,"tag":84,"props":679,"children":680},{"class":86,"line":106},[681],{"type":32,"tag":84,"props":682,"children":684},{"emptyLinePlaceholder":683},true,[685],{"type":37,"value":686},"\n",{"type":32,"tag":84,"props":688,"children":689},{"class":86,"line":132},[690,695,700,705,709,714],{"type":32,"tag":84,"props":691,"children":692},{"style":621},[693],{"type":37,"value":694},"with",{"type":32,"tag":84,"props":696,"children":697},{"style":91},[698],{"type":37,"value":699}," client.trace(",{"type":32,"tag":84,"props":701,"children":702},{"style":660},[703],{"type":37,"value":704},"name",{"type":32,"tag":84,"props":706,"children":707},{"style":621},[708],{"type":37,"value":652},{"type":32,"tag":84,"props":710,"children":711},{"style":121},[712],{"type":37,"value":713},"\"summarize_revenue\"",{"type":32,"tag":84,"props":715,"children":716},{"style":91},[717],{"type":37,"value":718},"):\n",{"type":32,"tag":84,"props":720,"children":721},{"class":86,"line":175},[722,727,731],{"type":32,"tag":84,"props":723,"children":724},{"style":91},[725],{"type":37,"value":726},"    result ",{"type":32,"tag":84,"props":728,"children":729},{"style":621},[730],{"type":37,"value":652},{"type":32,"tag":84,"props":732,"children":733},{"style":91},[734],{"type":37,"value":735}," llm.invoke(prompt)\n",{"type":32,"tag":84,"props":737,"children":738},{"class":86,"line":193},[739,744,749],{"type":32,"tag":84,"props":740,"children":741},{"style":91},[742],{"type":37,"value":743},"    client.log_metric(",{"type":32,"tag":84,"props":745,"children":746},{"style":121},[747],{"type":37,"value":748},"\"token_count\"",{"type":32,"tag":84,"props":750,"children":751},{"style":91},[752],{"type":37,"value":753},", result.usage.total_tokens)\n",{"type":32,"tag":33,"props":755,"children":756},{},[757],{"type":37,"value":758},"Cada trace aparece en la UI de LangSmith, con input\u002Foutput\u002Fmetadata completamente registrados. Si tienes múltiples versiones de prompts, puedes abrir una vista de comparación. La UI te muestra insights como \"el prompt v2 produce outputs un 8% más largo que v1 en promedio, pero su latencia es un 3% menor\".",{"type":32,"tag":33,"props":760,"children":761},{},[762],{"type":37,"value":763},"LangSmith también proporciona un playground — cambias el prompt y pruebas contra múltiples inputs con un click. Esto crea un loop rápido de feedback tanto para prototyping como para pruebas de regresión. Pero ojo: probar en el playground no reemplaza A\u002FB testing en production — solo proporciona un primer filtro.",{"type":32,"tag":40,"props":765,"children":767},{"id":766},"el-segundo-efecto-del-versionamiento-de-prompts-rollback",[768],{"type":37,"value":769},"El segundo efecto del versionamiento de prompts: rollback",{"type":32,"tag":33,"props":771,"children":772},{},[773],{"type":37,"value":774},"Cuando ocurre un deployment error, poder hacer rollback es crítico. En operaciones LLM, rollback significa volver a la versión anterior del prompt. Pero para hacerlo, necesitas un historial de versiones de prompts.",{"type":32,"tag":33,"props":776,"children":777},{},[778,780,786],{"type":37,"value":779},"El enfoque simple: guarda cada prompt en git en archivos separados (",{"type":32,"tag":80,"props":781,"children":783},{"className":782},[],[784],{"type":37,"value":785},"prompts\u002Fsummarization_v3.txt",{"type":37,"value":787},"). Tu script de deployment registra cuál versión está en production en un archivo de config:",{"type":32,"tag":73,"props":789,"children":793},{"code":790,"language":791,"meta":16,"className":792,"style":16},"# config\u002Fproduction.yaml\nprompts:\n  summarization: v3\n  classification: v2\n","yaml","language-yaml shiki shiki-themes github-dark",[794],{"type":32,"tag":80,"props":795,"children":796},{"__ignoreMap":16},[797,806,820,837],{"type":32,"tag":84,"props":798,"children":799},{"class":86,"line":87},[800],{"type":32,"tag":84,"props":801,"children":803},{"style":802},"--shiki-default:#6A737D",[804],{"type":37,"value":805},"# config\u002Fproduction.yaml\n",{"type":32,"tag":84,"props":807,"children":808},{"class":86,"line":97},[809,815],{"type":32,"tag":84,"props":810,"children":812},{"style":811},"--shiki-default:#85E89D",[813],{"type":37,"value":814},"prompts",{"type":32,"tag":84,"props":816,"children":817},{"style":91},[818],{"type":37,"value":819},":\n",{"type":32,"tag":84,"props":821,"children":822},{"class":86,"line":106},[823,828,832],{"type":32,"tag":84,"props":824,"children":825},{"style":811},[826],{"type":37,"value":827},"  summarization",{"type":32,"tag":84,"props":829,"children":830},{"style":91},[831],{"type":37,"value":118},{"type":32,"tag":84,"props":833,"children":834},{"style":121},[835],{"type":37,"value":836},"v3\n",{"type":32,"tag":84,"props":838,"children":839},{"class":86,"line":132},[840,845,849],{"type":32,"tag":84,"props":841,"children":842},{"style":811},[843],{"type":37,"value":844},"  classification",{"type":32,"tag":84,"props":846,"children":847},{"style":91},[848],{"type":37,"value":118},{"type":32,"tag":84,"props":850,"children":851},{"style":121},[852],{"type":37,"value":853},"v2\n",{"type":32,"tag":33,"props":855,"children":856},{},[857,859,865],{"type":37,"value":858},"Para rollback, cambias ",{"type":32,"tag":80,"props":860,"children":862},{"className":861},[],[863],{"type":37,"value":864},"summarization: v2",{"type":37,"value":866}," y disparas el deployment. Pero esto es manual y lento en un incident. Un enfoque más avanzado: usa un sistema de feature flags (LaunchDarkly, Unleash). Los flags te permiten cambiar la versión del prompt en runtime, sin desplegar código.",{"type":32,"tag":33,"props":868,"children":869},{},[870,872,879],{"type":37,"value":871},"Las prácticas de ",{"type":32,"tag":485,"props":873,"children":876},{"href":874,"rel":875},"https:\u002F\u002Fwww.roibase.com.tr\u002Fes\u002Ffirstparty",[489],[877],{"type":37,"value":878},"Primera Parte de Datos & Arquitectura de Medición",{"type":37,"value":880}," de Roibase entran aquí — necesitas conectar cambios de prompts con eventos downstream (conversión, churn) para basar tus decisiones de rollback en números. Si 6 horas después del deployment del nuevo prompt la tasa de churn sube un 4%, esa es la señal para rollback.",{"type":32,"tag":40,"props":882,"children":884},{"id":883},"edge-case-versionamiento-de-prompts-multiidioma",[885],{"type":37,"value":886},"Edge case: versionamiento de prompts multiidioma",{"type":32,"tag":33,"props":888,"children":889},{},[890],{"type":37,"value":891},"Si tu aplicación LLM funciona en múltiples idiomas (TR, EN, DE), necesitas mantener versiones separadas de prompts para cada idioma. Un prompt que funciona bien en inglés puede no dar el mismo tono en turco.",{"type":32,"tag":33,"props":893,"children":894},{},[895],{"type":37,"value":896},"Solución: organiza archivos de prompts por código de idioma:",{"type":32,"tag":73,"props":898,"children":900},{"code":899},"prompts\u002F\n  summarization\u002F\n    en_v3.txt\n    tr_v3.txt\n    de_v3.txt\n",[901],{"type":32,"tag":80,"props":902,"children":903},{"__ignoreMap":16},[904],{"type":37,"value":899},{"type":32,"tag":33,"props":906,"children":907},{},[908],{"type":37,"value":909},"Tu eval set también debe ser específico por idioma — en casos de prueba en turco, espera outputs en turco. Ejecuta A\u002FB testing por idioma, porque el comportamiento de usuarios turcos puede diferir del de usuarios en inglés. No olvides añadir segmentación por idioma en la agregación de métricas.",{"type":32,"tag":33,"props":911,"children":912},{},[913],{"type":37,"value":914},"Otro punto a considerar: la longitud de contexto varía por idioma — una oración en turco es en promedio un 12% más larga (en tokens). Esto es riesgo de superar el límite de tokens. Añade a tu pipeline de evaluación una verificación de token count, emite una alerta si superas el threshold.",{"type":32,"tag":40,"props":916,"children":918},{"id":917},"paso-práctico-configura-tu-primer-eval-set",[919],{"type":37,"value":920},"Paso práctico: configura tu primer eval set",{"type":32,"tag":33,"props":922,"children":923},{},[924],{"type":37,"value":925},"Para implementar el sistema descrito, el primer paso es un eval set minimal de 20-30 queries de usuarios reales. Abre tus logs de production, selecciona las queries más frecuentes, y para cada una define propiedades esperadas del output (precisión, tono, longitud).",{"type":32,"tag":33,"props":927,"children":928},{},[929],{"type":37,"value":930},"Luego configura Promptfoo o LangSmith, ejecuta tu prompt actual contra este set, obtén una puntuación baseline. Ahora haz un pequeño cambio en el prompt (por ejemplo, añade \"responde de forma breve y clara\"), ejecuta eval de nuevo y compara puntuaciones. Si no hay más del 5% de regresión, despliega el cambio.",{"type":32,"tag":33,"props":932,"children":933},{},[934],{"type":37,"value":935},"Cuando este ciclo se automatiza, tu velocidad de iteración de prompts se triplica. Porque ahora respondes \"¿es este cambio bueno o malo?\" no con suposiciones, sino con números.",{"type":32,"tag":937,"props":938,"children":939},"style",{},[940],{"type":37,"value":941},"html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}",{"title":16,"searchDepth":106,"depth":106,"links":943},[944,945,946,947,948,949,950],{"id":42,"depth":97,"text":45},{"id":63,"depth":97,"text":66},{"id":419,"depth":97,"text":422},{"id":593,"depth":97,"text":596},{"id":766,"depth":97,"text":769},{"id":883,"depth":97,"text":886},{"id":917,"depth":97,"text":920},"markdown","content:es:ai:versionamiento-de-prompts-y-pruebas-ab-disciplina-en-operaciones-llm.md","content","es\u002Fai\u002Fversionamiento-de-prompts-y-pruebas-ab-disciplina-en-operaciones-llm.md","es\u002Fai\u002Fversionamiento-de-prompts-y-pruebas-ab-disciplina-en-operaciones-llm","md",1783289078217]