cd ..
Godot Team 4 University of L'Aquila · CreateLab

PISCO

Play-based Insight and Social Cognition

A serious-game platform that turns clinical social-cognition assessments into a cosy pixel-art RPG with high-precision behavioural telemetry.

showcase.mp4
features.idx (8 features)
  1. overview
  2. what_i_did
  3. features/
  4. 01Dynamic Graph-Driven Experiment Execution Engine
  5. 02Sub-Millisecond & Physical Kinematic Telemetry Engine
  6. 03Battery of Standardized Cognitive Assessment Tasks
  7. 04Decoupled Multi-Modal Dialogue System
  8. 05Modular 6-Layer Composite Character System
  9. 06Autonomous Companion AI
  10. 07Context-Aware Directional Actionable System
  11. 08Production-Grade Networking & State Services
  12. gallery/

Project overview

PISCO is an innovative serious game and clinical research platform developed in collaboration with CreateLab at the University of L'Aquila (UNIVAQ), autism specialists, and cognitive neuroscience researchers. Designed to modernize cognitive evaluation and social cognition rehabilitation, PISCO transforms rigid, anxiety-inducing clinical test batteries into an inviting 2D top-down pixel-art RPG set within a virtual home.

Patients navigate their avatar, complete standardized neuropsychological protocols dynamically downloaded from a custom-made web portal DAG editor, and earn rewards to customize their home and care for a pet, while the engine transparently logs high-precision behavioral telemetry.

The platform bridges clinical rigorousness and playful engagement: psychologists define custom test flows visually on a React Flow web dashboard, while the Godot 4 client executes the graph dynamically, adapting task sequences and recording micro-behavioral metrics for diagnostic support.

What I did

As Lead Developer, System Architect, Gameplay Programmer, and 2D Pixel Artist, I designed and developed the entire Godot 4 game from scratch.

I architected the dynamic Directed Acyclic Graph (DAG) experiment parser (ExperimentManager) to execute React Flow protocols at runtime, implementing in-degree topological root discovery and dynamic conditional branching based on patient scores.

I implemented the standardized cognitive tasks (Theory of Mind, Baron-Cohen Faux Pas Recognition with skip logic, RMET, Emotion Attribution), engineered a sub-millisecond and physical DPI-normalized kinematic telemetry tracking engine (measuring independent reaction hesitation vs. response times and cursor trajectory length in centimeters), co-created the modular composite character system, programmed the companion pet AI, and integrated JWT RESTful cloud services with DTO serialization.

Team project · Lead Game Developer, System Architect, Gameplay Programmer, 2D Pixel Artist & Animator, Co-Project Coordinator

Features I implemented

01

Dynamic Graph-Driven Experiment Execution Engine

I architected a runtime Directed Acyclic Graph (DAG) execution engine inside ExperimentManager that dynamically interprets clinical protocols authored by psychologists in a React Flow web portal.

When a patient inputs their clinical session redemption code, the engine parses the raw JSON graph, builds adjacency dictionaries for edges and nodes, and executes in-degree root discovery to identify the starting node (the node with zero incoming edges). Tasks are dispatched dynamically using string-mapped task handlers, and conditional branching thresholds (Operator.GREATER_EQUAL, etc.) are evaluated based on patient score.

DAG experiment execution graph and root node discovery
Dynamic DAG parsing with in-degree root discovery and conditional edge evaluation
gdscript
func _prepare_experiment() -> void:
	var redeemed_flow: Dictionary = APIManager.redeemed_flow.flow.flow_json
	
	if not redeemed_flow:
		return
	
	var target_ids: Dictionary = {}
	
	# Map all edges and track every node that is a target (has incoming arrows)
	for edge: Dictionary in redeemed_flow[EDGES_KEY]:
		var source_id: String = edge[REACT_FLOW_KEY][SOURCE_KEY]
		var target_id: String = edge[REACT_FLOW_KEY][TARGET_KEY]
		
		# Save the target_id in a dictionary for fast O(1) lookups
		target_ids[target_id] = true
		
		if not edges.has(source_id):
			edges[source_id] = []
		edges[source_id].append(edge)
	
	# Map all nodes and find the one with 0 incoming arrows
	var redeemed_flow_nodes: Array = redeemed_flow[NODES_KEY]
	
	for node: Dictionary in redeemed_flow_nodes:
		var node_id: String = node[ID_KEY]
		nodes[node_id] = node
	
		# If this node's ID was never recorded as a target, it's the starting node
		if not target_ids.has(node_id):
			current_node_id = node_id


func _start_node() -> void:
	var current_node: Dictionary = nodes[current_node_id]
	var current_node_type: String = current_node[TYPE_KEY]
	
	# Add node record
	var current_node_record: NodeRecord = NodeRecord.new(current_node_id, current_node[TITLE_KEY], current_node_type, current_node[IS_EXERCISE_KEY])
	experiment_records[current_node_id] = current_node_record
	
	# Fetch the correct handler component dynamically!
	var handler: BaseExperimentTask = task_handlers.get(current_node_type)
	assert(handler != null, "No handler found for node type: %s" % [current_node_type])
	
	# Connect to its completion signal and start it
	handler.task_completed.connect(_on_task_completed, CONNECT_ONE_SHOT)
	handler.start_task(current_node)
In-degree root discovery and dynamic task dispatching in ExperimentManager
02

Sub-Millisecond & Physical Kinematic Telemetry Engine

To provide clinicians with diagnostic data beyond simple right/wrong answers, I engineered a high-precision dual-phase behavioral tracking suite composed of MouseDistanceTracker, FirstInputInterceptor, and Stopwatch.

The system measures reaction time (cognitive hesitation before initiating any mouse movement or keypress) independently from total response time (duration until final choice selection). Furthermore, cursor trajectory is normalized into physical centimeters: the engine queries the operating system's display density via DisplayServer.screen_get_dpi(), converting pixel vectors into real-world physical motion metrics that remain scientifically comparable across different monitor sizes and resolutions.

Kinematic telemetry data showing reaction time and trajectory curves
High-precision telemetry logging reaction hesitation and DPI-normalized cursor distance
gdscript
func _ready() -> void:
	# Get the monitor's actual DPI from the OS (Godot 4 syntax)
	_screen_dpi = DisplayServer.screen_get_dpi()
	
	# Fallback just in case the OS fails to report a valid DPI
	if _screen_dpi <= 0:
		_screen_dpi = 96.0

func _input(event: InputEvent) -> void:
	if not is_tracking:
		return
		
	if event is InputEventMouseMotion:
		# Get the raw distance moved in pixels during this tiny movement
		# event.relative is a Vector2 of how far the mouse moved on X and Y
		var pixels_moved: float = event.relative.length()
		
		# Convert pixels to inches (Pixels divided by Pixels-Per-Inch)
		var inches_moved: float = pixels_moved / _screen_dpi
		
		# Convert inches to centimeters (1 inch = 2.54 cm)
		var cm_moved: float = inches_moved * 2.54
		
		# Add it to our total
		total_distance_cm += cm_moved
DPI-normalized physical cursor distance calculation in MouseDistanceTracker
03

Battery of Standardized Cognitive Assessment Tasks

I implemented digital adaptations of standardized neuropsychological assessment protocols used in clinical cognitive research.

Each task handler inherits from BaseExperimentTask and manages dedicated question controllers, dynamic UI layouts, and clinical branching logic:

Reading the Mind in the Eyes clinical task UI with eye expression choices
Reading the Mind in the Eyes (RMET) assessment module
A gray cat sitting in the center of the screen with 'Downloading image...' text displayed below it
Cloud image loading indicator
  • Theory of Mind (ToM): Multi-character narrative scenarios assessing false-belief understanding and mental state attribution.
  • Baron-Cohen Faux Pas Recognition: Detects social blunder awareness with automated skip logic (bypassing follow-up questions if no faux pas is detected).
  • Reading the Mind in the Eyes (RMET): Displays standardized gaze crops streamed dynamically from cloud endpoints with format sniffing.
  • Emotion Attribution & Recognition: Open-ended and multi-choice emotional state categorization with keyword normalization.
04

Decoupled Multi-Modal Dialogue System

I built an asynchronous, queue-based dialogue system (DialogueController) capable of presenting multi-modal cognitive interactions.

The architecture separates dialogue presentation into modular UI components: typewriter-animated text boxes (TextBox), dynamic multi-choice selectors (ChoicesBox), free-form text input fields (TextInput), and remote image loaders (ImageBox). The controller supports @tool scripts for in-editor live previewing of dialogue layouts.

Multi-modal dialogue box with typewriter text and choice selectors
Modular dialogue controller rendering animated questions and choice selectors
05

Modular 6-Layer Composite Character System

I co-developed a modular 6-layer composite sprite character system (CompositePlayer) dividing the avatar into Body, Face, Hair, Shirt, Pants, and Shoes.

The system supports real-time gender switching and wardrobe customization through CustomPlayerScreen. Layer animations are synchronized across all directional BlendTree states in Godot's AnimationTree, ensuring seamless 4-directional locomotion without sprite alignment drift.

Composite character customization screen showing layered clothing options
6-layer composite sprite architecture supporting full character customization
06

Autonomous Companion AI

To reinforce the cosy, comforting atmosphere of the clinical environment, I programmed an autonomous companion pet (BobaTheCat).

The pet operates on a 3-state Finite State Machine (IDLE, FOLLOW_PLAYER, PET). Steering mechanics use distance hysteresis thresholds to prevent jittery start-stop oscillations when following the player. When the player pauses and interacts, Boba transitions into the petting state, playing custom purring animations and persisting across room transitions.

A gray cat following the female character inside her bedroom
Autonomous pet AI following the player with distance hysteresis steering
07

Context-Aware Directional Actionable System

I designed the Actionable framework to manage world interactions cleanly. Interactive props (beds, wardrobes, doors, desks) utilize area triggers that detect player proximity.

The system validates player facing angles using 2D vector dot products, ensuring interactions only fire when the avatar is actually looking at the object. Multiple overlapping actionables are sorted by Euclidean distance, showing a dynamic exclamation indicator over the primary target.

Actionable trigger areas with directional facing validation
Directional actionable detection with player facing-angle verification
08

Production-Grade Networking & State Services

I engineered a robust RESTful networking layer (APIManager) handling communication with the PISCO clinical cloud backend.

Requests utilize JWT Bearer authentication tokens, structured DTO serialization (GameExecutionDTO, RedeemedPathResponseDTO), and ISO-8601 UTC timestamp formatting. Upon session completion, experiment telemetry and answer records are batched and posted asynchronously, with automatic error retry handling and user-friendly toast notifications.

API manager network architecture and DTO data models
RESTful API client with JWT Bearer authentication and DTO serialization

Like what you see?

Get in touch to talk about how it was built.