KJS-Educational-Institute (1)

KJ'S Educational Institute KJ College of Engineering and Management Research Department of Engineering Science Project Title Social Sense AI: Bridging the Social Perception Gap Innovation in Education Technology-enabled learning environments Social Awareness & Accessibility Designing for inclusion AI & Assistive Tech Bridging perception with intelligent systems A dedication to inclusive innovation and research excellence.

[Audio] Project Team Project Facilitator 4 Vikas Raskar Sir · Batch �� 1 PROJECT FACILITATOR Vikas Raskar Sir Batch �� Team Roles (Integrated with visuals) Problem Statement Core Research Niranjan Matale Mahek Shaikh System Design & Workflow Component Architecture Shardul Khadatare Sanika Khomane Limitations Ethical Framework & Economic Analysis Omkar Kothare Samir Khatib Future Extensions Pranvi Lagad.

[Audio] SocialSense AI: Bridging the Social Perception Gap A Hybrid Distributed Social-Context Augmentation System for the Visually Impaired..

[Audio] Table of Contents An organized roadmap of this presentation's sections 1 2 3 4 2. Project Team Roles and contributors 4. Addressing the Visual-Social Perception Gap 1. SocialSense AI: Bridging the Social Perception Gap Research goals and 3. The Unseen Challenge: Social Communication for the Visually Impaired Overview and approach Problem definition and motivation context 5 6 7 8 8. AI at the Core: Emotion Inference with CNN 6. The SocialSense AI Workflow: From Pixels to Pulses Model design and 5. Innovative System Design: Distributed Hybrid Architecture 7. Key Components: Smart Integration, Tangible Feedback End-to-end pipeline inference strategy High-level system Hardware and interface architecture elements 9 10 11 10. Economic Feasibility and Future Horizons 9. Measurable Success: Performance & Ethical Considerations 11. SocialSense AI: A New Dimension of Social Awareness Cost, deployment, and next steps Evaluation metrics and ethics Conclusions and final remarks.

[Audio] The Unseen Challenge: Social Communication for the Visually Impaired Beyond Words: The Richness of Visual Cues Facial expressions & microexpressions Subtle eye contact & gaze shifts Silent emotional reactions Non-verbal personal space cues Human communication is deeply embedded in these visual signals, which are often inaccessible to the visually impaired..

[Audio] Addressing the Visual-Social Perception Gap Current Assistive Tech Limitations The Core Research Question Existing systems excel at obstacle Can visual emotional cues be effectively detection, navigation, and text-to-speech, translated into meaningful tactile signals but critically miss the social-emotional using a low-cost, distributed AI system? dimension. This question drives our innovative Some emotions are expressed silently. approach, aiming to empower visually impaired individuals with enhanced social Presence may not involve speech. awareness without relying on expensive Emotional shifts may lack immediate embedded smart glasses. auditory cues. Personal space violations can go unnoticed..

[Audio] Innovative System Design: Distributed Hybrid Architecture Smartphone AI Engine Local UI, connectivity, and lightweight Cloud or edge AI for heavy inference and preprocessing. model updates. Wearable Sensory Feedback Context-aware sensors and low-latency data Haptics and audio delivering clear, real-time fusion. cues. Decoupling for Efficiency We split roles between devices so each component does what it does best. Enhanced Power Efficiency: Wearables run longer by offloading heavy compute. Cost-Effectiveness: Leverages existing smartphones to reduce specialized hardware. Scalability & Maintainability: Modular updates to AI models and firmware simplify improvements..

[Audio] The SocialSense AI Workflow: From Pixels to Pulses 1 1. Visual Data Acquisition Smartphone camera captures real-time video frames. 2 2. Person Detection OpenCV + lightweight object detection identifies human presence. 3 3. Face Localization Facial regions are extracted and normalized for AI analysis. 4 4. Emotion Inference TensorFlow Lite CNN model predicts emotional state (Happy, Angry, Neutral). 5 5. Decision Encoding Emotions are translated into simplified signals (e.g., 'H' for Happy). 6 6. Wireless Communication Encoded signals are sent to an ESP�� via Bluetooth. 7 7. Haptic Feedback Generation Coin vibration motor provides tactile cues to the user. This comprehensive pipeline ensures real-time processing and effective tactile communication of social cues..

[Audio] Key Components: Smart Integration, Tangible Feedback Smartphone: The AI Brain ESP32 Microcontroller: The Communicator Vibration Motor: Tactile Language Leverages existing camera, onLow-cost, energy-efficient hub Converts electrical commands device TensorFlow Lite handling Bluetooth reception, into distinct tactile patterns processing, and Bluetooth command parsing, and signal (short pulse = Person, double = connectivity for real-time routing to actuators. Provides Happy, long = Angry), enabling inference and UI feedback. the reliable wireless link non-visual social cue Essential for complex visual between phone and wearable. communication. analysis and decision-making. Real-time Processing & Data Flow System Integration: Components Working Together OpenCV handles person/face Smartphone captures frames localization; TensorFlow Lite and runs OpenCV performs emotion inference; preprocessing + TensorFlow encoding and low-latency Lite inference ³ encodes Bluetooth transmission ensure decisions ³ sends via immediate haptic responses to Bluetooth to ESP�� ³ ESP�� social cues. actuates vibration motor for tactile feedback..

[Audio] AI at the Core: Emotion Inference with CNN Convolutional Neural Network (CNN) Focused Emotion Recognition Input: Cropped and normalized facial images. Our prototype prioritizes a Output: Probability distribution across predefined emotion classes. concise set of three core Confidence Threshold: Applied to reduce misclassification and improve reliability. emotions: Happy: Positive social engagement. Angry: Potential conflict or discomfort. Neutral: General presence or ambiguous state. This deliberate choice minimizes class overlap, enhances model accuracy, and reduces dataset requirements, leading to improved prototype stability..

[Audio] Measurable Success: Performance & Ethical Considerations Key Performance Parameters Ethical Framework: Prioritizing Privacy Inference Latency: Speed of AI processing. No Image Storage: Data processed in real-time, not Bluetooth Transmission Delay: Real-time signal retained. transfer. Local Device Processing: All computation occurs on Model Confidence Level: Accuracy of emotion the user's device. prediction. No Cloud Transmission: Eliminates external data Battery Life: Sustainability of the wearable device. exposure risks. User Comfort Feedback: Subjective experience of Assistive Guidance: Provides information, not tactile cues. authoritative judgment. These metrics ensure robust evaluation and User Privacy: At the forefront of all design decisions. continuous improvement of the SocialSense AI system..

[Audio] Economic Feasibility and Future Horizons Economic Analysis Hardware Cost: Approximately ¹����2¹����, making it significantly more affordable than embedded smart glasses. Software: Open-source tools minimize development and licensing costs. Scalability: Low cost enables broader adoption and accessibility for the visually impaired community. Multimodal Fusion Advanced AI Models Multi-Person Awareness Integrating audio cues with Transformer-based Detecting & differentiating visual data for richer context. architectures for more emotions from multiple nuanced emotion detection. individuals simultaneously. Edge AI Compression Adaptive Haptic Patterns Optimizing models for even Context-aware vibration more efficient on-device sequences for intuitive processing. feedback..

[Audio] SocialSense AI: A New Dimension of Social Awareness SocialSense AI is not a navigation tool; it is a distributed social-context augmentation framework that converts visual emotional information into tactile feedback for visually impaired individuals. By focusing on emotional-social awareness, SocialSense AI empowers visually impaired individuals to engage more fully and confidently in social interactions, enriching their daily lives..

[Audio] References IEEE References: Springer References: N. Yalçin and M. Y. N. P. S. Rajendran, P. H. Venkateswara and S. T. McDaniel and S. Alisawi, "Enhancing Krishnan, and D. J. Panchanathan, Domain Panchanathan, Haptic Social Interaction for Aravindhar, "Design Adaptation in Interfaces for the Visually Impaired: and Implementation of Computer Vision with Accessibility, Health, A Systematic Review of Voice Assisted Smart Deep Learning, and Enhanced Quality Real-Time Emotion Glasses for Visually Springer International of Life, Springer Recognition Using Impaired People Using Publishing, ����. International Smart Glasses and Google Vision API," Publishing, ����. (Supports CNN models Deep Learning," IEEE Proc. IEEE Int. Conf. and vision processing (Supports haptic Access, ����. Electronics and components) feedback and vibration Communication motor design) (ICECA), ����.