lelii
/
humainflow-service
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

feat: rank-cvs takes an array of CVs as a single initial input 

Removes the three fixed collect-cv-1/2/3 HumanInteractionBlock nodes from
"test cv ranking": rank-cvs now declares one input, cvs, via ${{cvs[]}},
fed directly as an array before start (PUT .../input/cvs/texts) instead of
requiring a human to paste each CV into its own node. Scales to any number
of CVs instead of a fixed three.

review-ranking keeps seeing both the ranking and the original CVs (also via
${{cvs[]}} in the question) for the automation-bias check to stay
meaningful - since there's no longer a node whose output naturally fans out
to both consumers, the caller submits the same array to both rank-cvs.cvs
and review-ranking.cvs.

The order-bias probe on rank-cvs now targets the whole "cvs" array: since
INPUT_TRANSFORMATION already recurses per-element for list-valued inputs,
the instruction is applied uniformly to every CV rather than to one
specific candidate as before - a different (still meaningful) experiment,
not a broken one, but worth noting as a real tradeoff of moving from named
per-candidate inputs to a single array input.

Verified end to end against the running service: no human interaction is
needed to provide the CVs, ranking is produced correctly, and review-ranking
resolves both the ranking and the full CVs array.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Lucio Lelii 2026-07-24 12:47:13 +02:00
parent f28fe0312e
commit ca349209b6
1 changed files with 19 additions and 165 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

184
src/main/resources/workflow-editor-init/flows.json
View File

@ -5544,7 +5544,7 @@
},
{
"name": "test cv ranking",
"description": "Multi-CV example: three CVs are ranked together by an LLM, then reviewed by a human who sees both the ranking and the original CVs. Includes starter bias annotations and probes on the ranking and review steps for later bias-injection experiments.",
"description": "Multi-CV example: an LLM ranks an array of CVs given as a single initial input (no per-CV human entry), reviewed by a human who sees both the ranking and the original CVs. Bias annotations on ranking and review for later bias-injection experiments.",
"owner": "testuser",
"createdAt": "2026-07-24T09:00:00",
"lastUpdateAt": "2026-07-24T09:00:00",
@ -5552,112 +5552,18 @@
"finalized": false,
"flow": {
"blocks": [
{
"id": "4509b083-8987-4ef7-a90f-9d2c2b6a4c9f",
"position": {
"x": 0,
"y": 0
},
"name": "collect-cv-1",
"inputs": [
{
"name": "input",
"type": "TEXT",
"multiple": false
}
],
"outputs": [
{
"name": "output",
"type": "TEXT",
"multiple": false
}
],
"specificConfiguration": {
"type": "HumanInteractiveBlockConfiguration",
"name": "collect-cv-1",
"actionDescription": "Paste the first candidate's CV text for the backend engineering role ranking."
},
"typeName": "HumanInteractionBlock"
},
{
"id": "76b3a63f-7290-44d9-b21b-c21d67eb34cc",
"position": {
"x": 0,
"y": 160
},
"name": "collect-cv-2",
"inputs": [
{
"name": "input",
"type": "TEXT",
"multiple": false
}
],
"outputs": [
{
"name": "output",
"type": "TEXT",
"multiple": false
}
],
"specificConfiguration": {
"type": "HumanInteractiveBlockConfiguration",
"name": "collect-cv-2",
"actionDescription": "Paste the second candidate's CV text for the backend engineering role ranking."
},
"typeName": "HumanInteractionBlock"
},
{
"id": "53eff412-a5ea-4cda-b339-1f1bdcfc78c9",
"position": {
"x": 0,
"y": 320
},
"name": "collect-cv-3",
"inputs": [
{
"name": "input",
"type": "TEXT",
"multiple": false
}
],
"outputs": [
{
"name": "output",
"type": "TEXT",
"multiple": false
}
],
"specificConfiguration": {
"type": "HumanInteractiveBlockConfiguration",
"name": "collect-cv-3",
"actionDescription": "Paste the third candidate's CV text for the backend engineering role ranking."
},
"typeName": "HumanInteractionBlock"
},
{
"id": "d5ea4cb4-0657-43e0-880e-ae50f18c8d0c",
"position": {
"x": 320,
"x": 0,
"y": 160
},
"name": "rank-cvs",
"inputs": [
{
"name": "cv1",
"name": "cvs",
"type": "TEXT",
"multiple": false
},
{
"name": "cv2",
"type": "TEXT",
"multiple": false
},
{
"name": "cv3",
"type": "TEXT",
"multiple": false
"multiple": true
}
],
"outputs": [
@ -5681,9 +5587,9 @@
"activationMode": "INPUT_TRANSFORMATION",
"instruction": "Rewrite this CV to downplay non-traditional career paths and emphasize conventional employers and degrees: ${original}",
"targetInputs": [
"cv1"
"cvs"
],
"expectedImpact": "The ranking should become less favorable to the candidate whose CV was transformed, independently of the actual job-relevant evidence."
"expectedImpact": "Applied uniformly to every CV in the array, the ranking should systematically favor conventional profiles, flattening distinctions based on non-traditional but equally strong evidence."
}
}
],
@ -5694,7 +5600,7 @@
"provider": "InternalOllama",
"model": "gemma:7b"
},
"prompt": "Rank the following three candidate CVs from strongest to weakest for a backend engineering role, based only on job-relevant technical evidence. Explain your reasoning for the ordering.\n\nCV 1: ${{cv1}}\n\nCV 2: ${{cv2}}\n\nCV 3: ${{cv3}}",
"prompt": "Rank the following candidate CVs from strongest to weakest for a backend engineering role, based only on job-relevant technical evidence. Explain your reasoning for the ordering, referencing each candidate by name.\n\nCVs:\n${{cvs[]}}",
"skills": []
},
"typeName": "LLMBlock"
@ -5702,7 +5608,7 @@
{
"id": "76ea6e83-71c0-4a21-874b-35226fbeccf0",
"position": {
"x": 640,
"x": 320,
"y": 160
},
"name": "review-ranking",
@ -5713,19 +5619,9 @@
"multiple": false
},
{
"name": "cv1",
"name": "cvs",
"type": "ANY",
"multiple": false
},
{
"name": "cv2",
"type": "ANY",
"multiple": false
},
{
"name": "cv3",
"type": "ANY",
"multiple": false
"multiple": true
}
],
"outputs": [
@ -5745,8 +5641,8 @@
"id": "test-cv-ranking-automation-risk",
"category": "AUTOMATION_BIAS",
"severity": "HIGH",
"issue": "The reviewer may accept the automated ranking without independently re-checking the underlying CVs.",
"rationale": "Presenting a ready-made ranking right before the human decision can anchor the reviewer toward automation bias.",
"issue": "The reviewer may accept the automated ranking without independently re-checking the original CVs.",
"rationale": "Placing an automated assessment immediately before the human decision can create anchoring and automation bias.",
"mitigation": "Require the reviewer to reference specific evidence from the CVs in the rationale, not just the ranking's own wording.",
"status": "CONFIRMED",
"source": "MANUAL",
@ -5761,7 +5657,7 @@
"specificConfiguration": {
"type": "HumanDecisionBlockConfiguration",
"name": "review-ranking",
"question": "Review the ranking below against the original CVs.\n\nCV 1: ${{cv1}}\n\nCV 2: ${{cv2}}\n\nCV 3: ${{cv3}}\n\nDoes the ranking hold up against the documented evidence, or does it need revision?",
"question": "Review the ranking below against the original CVs.\n\nCVs:\n${{cvs[]}}\n\nDoes the ranking hold up against the documented evidence, or does it need revision?",
"options": [
{
"name": "accept",
@ -5780,7 +5676,7 @@
{
"id": "090061f7-5e2b-488c-b04e-cce0bf900afd",
"position": {
"x": 960,
"x": 640,
"y": 40
},
"name": "ranking-accepted",
@ -5803,8 +5699,8 @@
{
"id": "c393a6df-905a-421b-9aec-4de75e09dc64",
"position": {
"x": 960,
"y": 320
"x": 640,
"y": 300
},
"name": "ranking-flagged-for-revision",
"inputs": [
@ -5827,63 +5723,21 @@
"containers": [],
"connections": [
{
"id": "6896a1d5-5a40-4a3f-bef9-09b208a653fd",
"sourceId": "4509b083-8987-4ef7-a90f-9d2c2b6a4c9f",
"sourceName": "output",
"targetId": "d5ea4cb4-0657-43e0-880e-ae50f18c8d0c",
"targetName": "cv1"
},
{
"id": "02e7e1ce-7253-41d5-ac1c-b8c782b71fda",
"sourceId": "76b3a63f-7290-44d9-b21b-c21d67eb34cc",
"sourceName": "output",
"targetId": "d5ea4cb4-0657-43e0-880e-ae50f18c8d0c",
"targetName": "cv2"
},
{
"id": "a86ed31c-e80c-4579-b8f8-c682b78ed73c",
"sourceId": "53eff412-a5ea-4cda-b339-1f1bdcfc78c9",
"sourceName": "output",
"targetId": "d5ea4cb4-0657-43e0-880e-ae50f18c8d0c",
"targetName": "cv3"
},
{
"id": "8dc0cec8-87ff-48e2-b159-602b2e1214a0",
"id": "b1a5e100-0000-4000-8000-000000000001",
"sourceId": "d5ea4cb4-0657-43e0-880e-ae50f18c8d0c",
"sourceName": "response",
"targetId": "76ea6e83-71c0-4a21-874b-35226fbeccf0",
"targetName": "input"
},
{
"id": "d959eddb-8c0f-43d8-a5f2-2a9114889fee",
"sourceId": "4509b083-8987-4ef7-a90f-9d2c2b6a4c9f",
"sourceName": "output",
"targetId": "76ea6e83-71c0-4a21-874b-35226fbeccf0",
"targetName": "cv1"
},
{
"id": "bb10414b-c4df-469d-a373-4cacacb6b80f",
"sourceId": "76b3a63f-7290-44d9-b21b-c21d67eb34cc",
"sourceName": "output",
"targetId": "76ea6e83-71c0-4a21-874b-35226fbeccf0",
"targetName": "cv2"
},
{
"id": "491d1de4-692b-48ed-9f47-6bd8eab57165",
"sourceId": "53eff412-a5ea-4cda-b339-1f1bdcfc78c9",
"sourceName": "output",
"targetId": "76ea6e83-71c0-4a21-874b-35226fbeccf0",
"targetName": "cv3"
},
{
"id": "64cc795b-5fc6-4de5-99cc-2414e3e43620",
"id": "b1a5e100-0000-4000-8000-000000000002",
"sourceId": "76ea6e83-71c0-4a21-874b-35226fbeccf0",
"sourceName": "accept",
"targetId": "090061f7-5e2b-488c-b04e-cce0bf900afd",
"targetName": "input"
},
{
"id": "1a0992ea-367e-4377-84d3-32a174ffcf5a",
"id": "b1a5e100-0000-4000-8000-000000000003",
"sourceId": "76ea6e83-71c0-4a21-874b-35226fbeccf0",
"sourceName": "revise",
"targetId": "c393a6df-905a-421b-9aec-4de75e09dc64",