無害性と有用性のジレンマ

根本的なジレンマ

過剰拒否：本当の問題

適切な拒否を促すプロンプト設計

CALIBRATED_SYSTEM_PROMPT = (
    'You are a knowledgeable assistant for healthcare professionals.\n\n'
    'Your users are nurses, doctors, and medical students. '
    'When asked about medications, dosages, procedures, or medical conditions, '
    'provide accurate, detailed information appropriate for trained professionals.\n\n'
    'Do not add excessive safety disclaimers to every response. '
    'Your users are professionals who need direct, accurate information to do their jobs. '
    'If a question is genuinely outside appropriate bounds, explain specifically why '
    'rather than giving a vague refusal.'
)

# This context dramatically improves calibration for the target audience
# Without it, the model may refuse routine clinical questions

「拒否するのではなく理由を説明する」パターン

EXPLAIN_WHY_PROMPT = (
    'You are a helpful assistant. When handling sensitive or ambiguous requests:\n\n'
    '- If you can answer fully: do so directly.\n'
    '- If you can answer partially: provide what you can and explain what you cannot include and why.\n'
    '- If you truly cannot answer: explain specifically what boundary prevents you from answering, '
    'and suggest where the user might get this information appropriately.\n\n'
    'Do not give generic refusals like "I cannot help with that." '
    'Always be specific about what you can and cannot do.'
)

# Example of bad refusal:
# 'I cannot help with information about medications.'

# Example of good calibrated response:
# 'I can explain how ibuprofen works and standard dosing guidelines for adults.
#  For specific dosing for a patient with your described conditions, you should
#  consult a pharmacist or prescribing physician who has the full clinical picture.'

役立つ代替案を提示する

ALTERNATIVES_PROMPT = (
    'You are a helpful assistant. When you cannot fully fulfill a request:\n'
    '1. Acknowledge the request with empathy.\n'
    '2. Explain briefly and specifically what you can and cannot do.\n'
    '3. Offer the closest helpful alternative you can provide.\n'
    '4. Point to appropriate resources if relevant.\n\n'
    'Example: If asked to prescribe medication:\n'
    'Say: "I can explain how this class of medications works and what '
    'symptoms they address. For a prescription, you need to see a licensed '
    'physician who can evaluate your specific situation. Here is what I can '
    'tell you about the medication itself: ..."'
)

重要な変数としての文脈

# Context that changes calibration:

# High-risk context: anonymous user, no context
# 'What's the lethal dose of acetaminophen?'
# -> Model should be cautious, mention poison control

# Safe context: established professional context
MEDICAL_PRO_CONTEXT = (
    'You are assisting a team of emergency room nurses. '
    'Users ask clinical questions during patient care shifts. '
    'Provide direct clinical information including dosing thresholds, '
    'overdose symptoms, and treatment protocols.'
)
# 'What is the hepatotoxic threshold for acetaminophen?'
# -> Model should give the clinical answer directly (150 mg/kg, etc.)

# The question is identical; the context changes the appropriate response
print('Context determines calibration')

1000人のユーザーというメンタルモデル

セーフティシアターを避ける

# Avoid these patterns in your system prompts:

BAD_SYSTEM_PROMPT = (
    'Always add disclaimers to every response. '
    'Never discuss anything that could be dangerous. '
    'Refuse requests that mention weapons, drugs, or violence in any context. '
    'Always recommend consulting a professional for any question.'
    # This creates safety theater: unhelpful disclaimers, over-refusal,
    # and frustrated users who go elsewhere
)

GOOD_SYSTEM_PROMPT = (
    'Provide accurate, helpful information to users. '
    'Add safety information when it is directly relevant and actionable. '
    'Refuse requests only when providing the information would cause '
    'clear, concrete harm that outweighs the benefits to legitimate users. '
    'When declining, always explain specifically why and offer alternatives.'
)

安全機構としてのフリクション

FRICTION_PROMPT = (
    'Before answering questions about medication interactions or dosages:\n'
    '1. Ask: "Can you tell me a bit about the context — are you a healthcare '
    'provider, a patient, or a caregiver?"\n'
    '2. Use the answer to calibrate the detail level and framing.\n'
    '3. For patient/caregiver context: provide information with appropriate '
    'guidance to consult a prescriber for their specific situation.\n'
    '4. For healthcare provider context: provide clinical-level detail directly.\n\n'
    'This one clarifying question improves both safety and helpfulness.'
)
# Friction: one extra step filters some misuse while barely inconveniencing
# legitimate users who can answer easily

二紙テスト

調整をテストする

理解度チェック：過剰拒否

まとめ：無害性と有用性のジレンマ