Experience
Return Zero Jul. 2024 - Present
Research Engineer
- Researched and developed state-of-the-art speech-to-text engines based on NVIDIA Parakeet architecture.
- Developed LLM-based readability evaluation system with 100M+ sentence dataset.
- Led end-to-end development of LLM-powered features for Callabo (AI meeting transcription service).
- Managed research-to-production pipeline including model fine-tuning and inference optimization.
NCSOFT Graphics AI Lab. Feb. 2021 - Jun. 2024
Machine Learning Researcher
- Text-to-Texture Generation (Dec 2023-Jun 2024): Built infrastructure and web application evaluating 5+ generative models for natural texture generation on 3D objects.
- Real-time Digital Human Facial Animation (Feb 2022-Nov 2023): Built production API server generating persona-aware facial expressions with sub-100ms response time.
- Speech-driven 3D Facial Animation (Feb 2021-Jan 2022): Designed PyTorch-based framework for deep learning model training, reducing setup time by 60%.
Naver Clova Jul. 2020 - Aug. 2020
Machine Learning Engineer, Intern
- Contributed to multi-scale depth estimation network development for 3D map reconstruction.
Naver Webtoon Research Sep. 2018 - Feb. 2019
Machine Learning Researcher, Intern
- Built annotation tool with real-time inference and dataset management for 2D bounding box support.
- Designed Visual QA task with multi-modal network for internal workshop validation.
SW Maestro 7th Jul. 2016 - Aug. 2018
- Garnered acclaim as a top team, showcased at the "100+ Conference" by the Ministry of Science and ICT, highlighting exceptional project development and execution.
Projects
Text-to-Texture Project
- Developed a scalable infrastructure, including model serving capabilities.
- Spearheaded the development of a Proof-of-Concept (PoC) web application using React.js and three.js, along with a FastAPI server.
- Conducted extensive research and experimentation on models for generating natural textures on 3D objects.
Digital Human Project
- Led the development of a real-time and responsive facial expression generation model and system for digital humans.
- Pioneered the generation of dynamic facial expressions and persona integration in conversational scenarios, enhancing digital human interactivity.
- Orchestrated the creation of highly detailed facial animations, capturing nuanced expressions tailored to unique character traits.
- Architected a high-performance API server with sub-100ms response time, optimizing it for diverse inputs in facial animation generation.
3D Facial Animation Generation Model Research
- Implemented advanced preprocessing and alignment pipelines for superior motion and facial capture data quality.
- Devised and deployed a PyTorch-based framework for deep learning model training, reducing setup time by 60%.
- Conducted innovative research on deep learning models for speech-driven 3D facial animation generation.
Webtoon Annotation Tool
- Engineered advanced annotation tools for efficient collection of domain-specific raw data.
- Developed a sophisticated analysis tool for cross-validating annotations, enabling comprehensive tracking of productivity, task difficulty assessment, and model inference comparison.
MariaDB Scalable Lock Manager
- Improved InnoDB storage engine performance by implementing a lock-free design and reusable object pool for the global lock mechanism in the transaction record manager.
Memento: Information Aggregation and Entity Linking System
- Built a scalable crawling system to aggregate articles and information from multiple sources.
- Implemented vector embedding and clustering system to link information to entities and events with continuous expansion capability.
DeepCheck: Deep Learning-Based Face Recognition Attendance System
- Developed a face recognition model optimized for low-resolution, crowded environments.
- Built a cross-platform application and API server using React Native for efficient model inference.
Education
Hanyang University Mar. 2019 - Feb. 2021
M.S. in Computer Science
Seoul, Korea
Advisor: Jongwoo Lim
Research on Multi-Object Tracking with deep learning-based object detection and tracking algorithms, proposing novel model architectures optimized for fisheye lens environments.
Awards
- Google Machine Learning Challenge Korea 5th place 2017
- Top start-up certification SW Maestro 7th 2017
- ACM-ICPC Asia Daejeon Regional Contest 16th place 2015
Publications
Object detection in fisheye lens environment
Image Processing and Image Understanding, 32nd Workshop, 2020
Object counting using object detection and re-identification in video sequence
Image Processing and Image Understanding, 32nd Workshop, 2020
Collaborative Training of Balanced Random Forests for Open Set Domain Adaptation
arXiv, abs/2002.03642, 2020