AI agents · Final Year Project, IBA
AI Workflow Automator
One web platform with two AI tools. A browser agent turns plain-English instructions into real clicks and keystrokes across web systems. PaperMark grades scanned exam scripts and notebooks against a rubric, with the teacher approving every mark.
- Home
- Announcements
- Content
- Grades
- read accessibility tree
- click link “Announcements”
- fill title, message
- click button “Post”
- done · 11 iterations · 31 s
The challenge
Faculty spend hours on computer work that doesn’t need a teacher: posting announcements, uploading material, copying marks between systems, emailing students and grading stacks of scanned scripts.
Traditional RPA bots break when a page changes. AI chat assistants can explain what to do, but can’t actually do it.
The solution: Browser Agent
My focus
- InstructionFastAPI service
- LLM picks a toolsmall, typed tool set
- Action runsPlaywright · Chromium
- Fresh snapshotaccessibility tree
Agentic loop
An instruction goes to a FastAPI service. An LLM chooses from a small, typed set of tools. Actions run on a live Chromium browser through Playwright. A fresh page snapshot comes back after every action until the task is done.
Reads structure, not pixels
The agent works from the page’s accessibility tree, so it survives cosmetic redesigns that would break a pixel- or selector-based bot.
Human in the loop by design
It pauses for CAPTCHAs, one-time passwords and judgement calls, asks the user in the browser and dashboard, then continues from the same state.
Built for real conditions
History pruning for long runs, parallel form filling (a five-field form in one step), and automatic backoff and retry when rate-limited.
Model-agnostic
Swaps between OpenAI, Anthropic and Google models per task.
The solution: PaperMark
Team-built grading pipeline
Eight stages, from a stack of scanned scripts to marks in the LMS, with the teacher approving every mark.
- 01Upload
- 02OCR
- 03Segment answers
- 04Match to student
- 05Crop answers
- 06AI marking
- 07Teacher review human
- 08Export CSV/XLSX
Answer cropping handles answers that span two pages.
Results
- 9 / 10real workflows
Real LMS/UMS workflows completed end-to-end, unattended.
- 31 s11 iterations
Posting an LMS announcement: the fastest and cheapest of the models tested.
- ~1 daywas about a week
Grading 90 scripts × 8 questions.
- 92.3%OCR accuracy
Character-level OCR accuracy.
- ~1.05marks from a human grader
Average difference between AI marks and a human grader.
- 60%accepted in one click
Of suggested marks, accepted with one click.
What this shows
I can design and ship AI agents that do real work in real systems, safely, with a human approving what matters. The same approach applies to any business that lives in web portals: data entry across systems, form filing, report collection, back-office admin.
Free automation review
Got a process that lives in web portals?
Describe it in a few lines. I’ll reply with how I’d automate it and what it should save.