Next Gen Admin
Cluster B: Innovation in Exam Administration
AI use in commercial vocational examinations
The use of generative artificial intelligence (AI) in commercial workplaces has the potential to transform vocational education and training. It creates new requirements for trainers and trainees, who must develop and apply competencies in dealing with digital technologies. The introduction of AI also requires awareness of legal frameworks that ensure the responsible use of technology without compromising ethical standards or data-protection requirements. Three issues can be identified as particularly relevant for the vocational education and training system:
- Obligation to foster AI competence: The statutory obligation to foster AI competence requires trainers to convey well-founded knowledge about artificial intelligence to learners and thereby support digital education.
- High-risk systems: AI systems used to assess learning outcomes or to determine access to educational pathways, for example through automated applicant screening, are classified as “high risk”. They are subject to strict requirements concerning transparency, human oversight, and data quality, thereby creating new responsibilities for examination administration.
- General labelling obligation: The use of AI systems is subject to a general labelling obligation. When trainees interact with generative AI, this must be clearly recognisable. Consequently, AI-generated content must be identifiable as such in order to prevent deception.
5 examined IHK sites
Ludwigshafen · Erfurt · Frankfurt · Hannover · Offenbach
5 transcribed group discussions
Interviews and workshops with examination administration
- Nominal Group Technique (approx. 90 min)
- Analysis of the data material: qualitative content analysis according to Kuckartz
- 3 independent coders → intercoder reliability: very high agreement (Krippendorff’s α = 0.82)
AI-related effects on competencies, examination organisation, assessment, and legal boundaries.
Point of departure
- AI is becoming part of commercial work
- Generative AI is changing tasks, value creation, and workplace assistance systems
- New requirements are emerging for trainers and trainees
Examination administration as a governance instance
It must address three questions:
- Which AI-related competencies need to be assessed?
- In which examination components can AI provide support?
- How can examinations remain centrally regulated, fair, and legally sound?
Three requirements in the AI regulatory context
- Foster AI literacy: AI competence becomes an object of vocational training
- Address high-risk uses: assessment, grading, admission, and screening are particularly sensitive areas
- Label AI use: AI-generated content and individual performance must remain distinguishable
(Aligned with the EU AI Act)
Development field 1:
Reconfigure examination formats
From product-oriented formats towards process-based transfer tasks and professional discussions.
Development field 2:
Standardise integrity standards
Standardised declarations of independent work instead of overly detailed AI-control mechanisms.
Development field 3:
Integrate AI into examinations
First strengthen administrative processes, then pilot evaluative AI applications.
Finding 1: AI competencies are becoming an integral part of vocational action competence.
AI literacy and critical reflection are becoming fundamental requirements for examination stakeholders.
Finding 2: AI offers relief potential in core administrative processes.
Especially in organisation, allocation, communication, and documentation.
Finding 3: In practice, the labelling obligation is considered difficult to implement on a sustained basis.
Transparent evidence of AI use is difficult to realise, both technically and didactically.
Finding 4: Technological dynamics collide with systemic inertia.
Slow regulatory and curriculum-development processes hinder the integration of innovative AI applications.
AI Strategy Consulting: The Trendslop Effect
The growing use of large language models (LLMs) such as ChatGPT promises to summarise complex information and produce polished strategic recommendations at lightning speed. However, an empirical study by Angelo Romasanta, Llewellyn D. W. Thomas and Natalia Levina shows that LLMs exhibit systematic biases in strategic reasoning. They follow fashionable buzzwords rather than the specific circumstances of an enterprise – a phenomenon the authors call “Trendslop”.
Research Design
In their investigation, the authors tested seven leading LLMs on seven fundamental strategic tensions. The models were asked to choose between two alternatives. Both the corporate contexts (e.g. start‑up vs. conglomerate) and the prompts (open question vs. pro‑/con‑instructions) were varied. The seven tensions were:
- Exploration vs. Exploitation
- Centralisation vs. Decentralisation
- Short term vs. Long term
- Competition vs. Collaboration
- Radical vs. Incremental
- Differentiation vs. Commodification
- Automation vs. Augmentation
Key Findings
The analyses revealed that, across most tensions, LLMs consistently favour the same option, regardless of the context and specific wording of the question. The most important biases are summarised below.
Differentiation > Commodification
Almost all LLMs advise companies to differentiate through unique offerings and brands rather than competing on low costs. In doing so, the models overlook the well‑established strategy of cost leadership.
Augmentation > Automation
The systems nearly always favour augmenting human labour with AI over fully automating processes, reflecting the positive connotation of “augmentation” in contemporary discourse.
Long term > Short term
LLMs prefer long‑term strategies – even in situations where short‑term measures may be critical for survival. This reinforces the trend toward long‑range thinking.
Exploration vs. Exploitation
Only in this tension do notable differences emerge across models. ChatGPT, for example, tends to favour exploration, whereas other models lean more strongly toward exploiting existing capabilities.
The Hybrid Trap
If not forced to choose, the models often recommend doing “everything at once” – for example, differentiation and cost leadership. Such hybrid strategies are considered risky in strategic management and lead to unclear priorities.
Causes of Biases
LLMs are trained on vast quantities of publicly available texts. Terms such as “differentiation”, “augmentation” or “collaboration” carry positive associations in contemporary business discourse, while “commodification” or “hierarchy” are laden with negative connotations. Because LLMs select words based on their statistical attractiveness, they reproduce these cultural biases. Additionally, they mainly consume modern management narratives and ignore classic strategy theories such as Michael Porter’s advocacy of cost leadership.
Recommendations for Using LLMs
- Use LLMs as idea generators: The models are useful for generating alternatives, risks and stakeholder perspectives, but should never replace the final decision.
- Confront biases intentionally: Prompt the model explicitly to develop strong arguments for the less fashionable options (e.g. commodification, short time horizons).
- Question hybrid recommendations: When the model advocates mixed strategies, treat this as a warning signal. Develop separate risk/benefit analyses for each option.
- Context alone is insufficient: Detailed corporate descriptions only modestly reduce the biases; the model remains shaped by its trend‑driven preferences.
- Preserve strategic judgement: Executives must retain decision authority and remain aware of the cultural imprint embedded in LLMs.
Legal Foundations for AI in Vocational Education
The legal opinion by Heckmann, Paschke and Rachut examines the diverse sources of law that govern the deployment of AI within vocational education. From constitutional provisions and EU regulations to data protection and copyright, exam officials and training providers must address a multitude of requirements. This overview systematises the most important articles and sections and shows how they feed into examination administration.
Legal Foundations
Basic Law & Vocational Training Law
The Basic Law (Grundgesetz) guarantees, in Article 12(1), the freedom to choose and practise an occupation and, in Article 3(1), the equal treatment of all candidates. The Crafts Code and the Vocational Training Act (BBiG) stipulate that examinations must accurately reflect training standards. Tasks may not unfairly disadvantage anyone and must be legitimised by statute (principle of materiality). These norms form the constitutional framework for designing AI‑supported examinations.
EU AI Act
The regulation defines AI systems in Article 3 and distinguishes, among others, \"high‑risk\" applications (Chapter II). Articles 6 and 8 set requirements for risk management, transparency and human oversight. Article 52 describes labelling obligations when generative AI produces content. These provisions govern the use of AI assistants in examinations and oblige providers and operators to maintain documentation and compliance.
Data Protection & GDPR
The General Data Protection Regulation (GDPR) sets out, in Article 5, the principles of data processing (lawfulness, purpose limitation, data minimisation). Article 6 explains when processing is permissible, and Article 25 mandates \"privacy by design\". For AI examinations, this means that personal data must be collected sparingly, processed within the EU and safeguarded against unauthorised access. Data subjects have rights to access, rectification, erasure and objection (Articles 15–17 GDPR).
Copyright, Contract & Liability Law
The Copyright Act (UrhG) protects creative works and governs the relationship between original and AI‑generated content. The use of training data and the release of examination material must respect third‑party rights. Contract law (German Civil Code) secures the relationship between training providers and AI vendors. Liability issues must be clarified: who is liable for system errors? The opinion urges clear contractual provisions and limits on liability.
Risk Classes & Roles
The opinion proposes classifying AI systems according to their critical impact: peripheral (e.g. calendar services), low (language models for inspiration), medium (assistance in exam preparation), high (automated marking and admission decisions) and very high (fully automatic decision making). The higher the risk, the stricter the duties for providers and operators – including transparency, auditing, human control and documentation obligations.
Examination Principles & Fairness
Examinations should reflect the candidates’ own performance. Examination regulations must specify permissible aids and uphold equal opportunities. The use of AI detectors is rejected, as it relies on statistical probabilities and is prone to discrimination. Instead, the opinion advocates task formats that demand context, transfer and argumentative justification. Examination bodies should define clear procedures for suspected manipulation based on personal hearings rather than purely technical tests.
Recommendations & Governance
- Ensure legal embedding: Explicitly anchor AI examinations in examination regulations and refer to the relevant articles of the AI Act, GDPR and the Basic Law.
- Transparency and labelling: Require providers to disclose training data and functionalities (Article 52 AI Act) and clearly label the use of AI to candidates and examiners.
- Risk‑adapted procedures: Define specific workflows for each risk class – from risk analysis and conformity assessment to human oversight for highly critical applications.
- Privacy‑oriented architecture: Implement \"privacy by design\" (Article 25 GDPR), store data within EU infrastructures and minimise personal data. Establish clear consent processes.
- Strengthen legal expertise: Train examination administrators in relevant legal areas (AI Act, GDPR, Copyright Law, liability law) so they can make legally compliant decisions.
- Integrity‑oriented tasks: Design exams so that reflective, argumentative and contextual abilities are paramount. Evaluate not only the result but also the process.