Production-Ready Applied Deep Learning

Tomasz Palczewski*, Jaejun Lee*, Lenin Mookiah*
- Packt Publishing; Date of Publication: August 31, 2022
*Equal contribution

Co-Informatic Generative Adversarial Networks for Efficient Data Co-Clustering

Jaejun Lee, Hyun Chul Lee, Tomasz Palczewski
- International Patent (WO/2021/066530) - Date of Patent: April 8, 2021
- U.S. Patent (20210097372) - Date of Patent: Januray 4, 2021

CI-GAN: Co-Clustering By Information Maximizing Generative Adversarial Networks

Jaejun Lee, Hyun Chul Lee, Tomasz Palczewski
- 2021 IEEE International Conference on Multimedia and Expo (ICME)
- Download version with appendix

Howl: A Deployed, Open-Source Wake Word Detection System

Raphael Tang*, Jaejun Lee*, Afsaneh Razi, Julia Cambre, Ian Bicking, Jofish Kaye, Jimmy Lin
- Proceedings of Second Workshop for NLP Open Source Software (NLP-OSS)
*Equal contribution

Showing Your Work Doesn't Always Work

Raphael Tang, Jaejun Lee, Ji Xin, Xinyu Liu, Yaoliang Yu, Jimmy Lin
- Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL 2020)

DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference

Ji Xin, Raphael Tang, Jaejun Lee, Yaoliang Yu, Jimmy Lin
- Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL 2020)

Honkling: In-Browser Personalization for Ubiquitous Keyword Spotting

Jaejun Lee, Raphael Tang, Jimmy Lin
- Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)

What Would Elsa Do? Freezing Layers During Transformer Fine-Tuning

Jaejun Lee, Raphael Tang, Jimmy Lin
- arXiv: 1911.03090
- Download revised version with crosslingual language model analysis

Information Retrieval Meets Scalable Text Analytics: Solr Integration with Spark

Ryan Clancy, Jaejun Lee, Zeynep Akkalyoncu Yilmaz, Jimmy Lin
- Proceedings of the 42nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2019)

Universal Voice-Enabled User Interfaces using JavaScript

Jaejun Lee, Raphael Tang, Jimmy Lin
- Proceedings of the 24th International Conference on Intelligent User Interfaces: Companion (IUI 2019)

JavaScript Convolutional Neural Networks for Keyword Spotting in the Browser: An Experimental Analysis

Jaejun Lee, Raphael Tang, Jimmy Lin
- arXiv: 1810.12859

Research Scientist

Epson
Mar 2023 - current

- Conducting research to develop a novel interpretable 2D object and attribute detection system designed for non-expert users
 - Invented a two-stage adapter-based domain adaptation method that utilizes a contrastive loss focused on hard-negative samples
 - Enabling effective verification of the model’s representations by establishing a strong understanding of primitive concepts
- Proposed and developed a prompt-based 2D keypoint detection pipeline that eliminates keypoint-specific training needs while remaining robust in multi-instance scenarios

Lead Research Scientist

RoboEye.ai
Jul 2021 - Mar 2023

- Developed data-driven computer vision and robotics solutions integrating latest AI techniques
 - C++ robotics solution for bin-picking tasks using ROS, Qt5, ZeroMQ, Protobuf, OpenCV, and PCL
 - Real-time (<1 sec) 6D pose estimation pipeline (depth completion + instance segmentation + pose estimation + pose refinement + detection filtering)
 - Fully automated online model training system using PyTorch, NVIDIA Isaac Sim, OpenCV, and AWS
 - Object detection performance tracking system using AWS, Docker, W&B, Django, and PostgresSQL
- Deployed 50+ bin-picking systems that run 24/7 with minimal human interventions
- Coordinated an R&D team of 20+ engineers by monitoring progress using Jira
- Reviewed customer requests to construct project requirements (scope, cost, and timelines)
- Set up internal protocols for a timely deployment of systems to Chinese factories

Research Scientist

RoboEye.ai
Mar 2020 - Jul 2021

- Implemented state-of-the-art 6D pose estimation algorithms in PyTorch
- Developed a robotics solution for bin-picking tasks in C++ using ROS, Qt5, ZeroMQ, OpenCV, and PCL
- Set up quality assurance and data protection process through GitLab CI using Docker and GCP

Research Scientist - Visual Display Intelligence Lab

Samsung Economics Research Institute
Apr 2019 – Mar 2020

- Invented the first generative modeling approach for co-clustering; jointly learns disentangled representations of dual data dimensions and their underlying interrelation in the correlation space
- Implemented user-centric TV program recommendation by analyzing watch history
- CI-GAN : Co-Clustering By Information Maximizing Generative Adversarial Networks (ICME 2021)
- Co-informatic generative adversarial networks for efficient data co-clustering (Patent ID: US20210097372A1, WO2021066530A1)

Software Engineer - Dynamic Ads Infra *

Meta
Jan 2018 – Apr 2018

- Dynamic Ads Infra Team supports product level advertisements by applying the k-nearest neighbor algorithm to define the target audience
- Redesigned the system to consider a wider range of products at an earlier stage; increased the click-through rate
- Refactored existing code base to reduce latency and memory usage

Software Engineer - Core Infrastructure Team *

Uber
May 2017 – Aug 2017

- Complex Data Processing/Spark Team is focusing on improving developers experiences with Spark within Uber as well as Spark community by implementing various services relating big data
- Integrated Tensorflow and TensorflowOnSpark on Uber's Spark-based notebook environment for data scientists
- Benchmarked performance and evaluated stability of TensorFlowOnSpark on Uber infrastructure
- Transformation of existing MLlib job into TensorFlow job on Spark reducing training time from 33 hours to 3 hours
- Investigating the integration of Tensorflow on various resource managers to provide better-developing environments for deep learning
- Investigating various Deep learning tools for Spark recommending the right tools to use for various use cases

Software Engineer - Central Technology Organization *

Zynga
Aug 2016 – Dec 2016

- Developed new architecture for search on the internal webpage; improved data integrity led to a 30% increase in search usage (Amazon ElasticSearch, Amazon Kinesis Streams and Amazon SQS)
- Maintained the internal web service that organizes technical products developed by Zynga
- Implemented web crawler that collects data from various documentation archives allowing developers to share integration guide internally (Angular.js, Node.js and MongoDB)
- Improved the backend system with daily data synchronization job using Java and Couchbase

Software Engineer - Emerging Technologies Team *

SAP
Jan 2016 – Apr 2016

- The Emerging Technologies group is a dedicated team of skilled technical, business, and design experts focused on innovating: mobile, IoT, web, web of things, edge computing, in-memory computing, extreme transaction processing, distributed data and analytics' solutions that extend SAP's leading position
- Designed and developed SQLA backend system that supports OData with distributed architecture
- Implemented a web application with extreme transaction processing using jQuery, SAP UI5 and SQLA
- Developed a tool for tracking employee traffic using Node.js and Reely Active Bluetooth sensors
- Integrated an automated testing tool (Robot Framework) to reduce QA cycle from 3 days to 4 hours

Full Stack Developer *

Mozzaz Corporation
May 2015 – Aug 2015

- Mozzaz delivers care solutions for individuals facing behavioral and mental challenges. Care Providers develop and administer treatment programs, and monitor patient progress using the company's innovative mobile solutions
- Developed a hybrid app with web development tools using Cordova; Back-end development with C#
- Released new data analysis tool using Twilio API, Fitbit API, Chart.js and AngularJS

* Co-op program

Howl: A Deployed, Open-Source Wake Word Detection System

University of Waterloo
Mar 2020 - Oct 2020        Github   YouTube

- Wake word detection modeling for Firefox Voice, supporting open datasets like Google Speech Commands and Mozilla Common Voice
- Howl represents the first fully productionized, open-source wake word detection toolkit with a web browser deployment target
- our deployed model has enabled Firefox Voice to provide a completely hands-free experience to over 8,000 users in the 9 days since its launch in August 2020 ¹
- experience Firefox Voice, the intelligent virtual assistant provided by Mozilla!

Honkling: In-Browser Personalization for Ubiquitous Keyword Spotting

University of Waterloo
May 2018 - Aug 2019        Github

- Implemented keyword spotting with convolutional neural networks in pure JavaScript that runs in any standards-compliant browser
- Applied fine-tuning based accent adaptation and its efficiency in the browser
- experience honkling, a novel in-browser keyword spotting system!

Composable Deep Neural Networks

University of Waterloo
Jan 2019 - Apr 2019        Github

- Presented Composing algorithm for neural networks; dynamically constructs a classifier for a changing set of targets
- Composed models require less computations as they skip unnecessary calculation but suffer from a decrease in performance
- Studied the impact of loss functions in Composing algorithm

Performance Evaluation of Solr on Large Scale Document Collections

University of Waterloo
Sep 2018 - Apr 2019        Github

- Evaluated performance of Solr on large document collections by querying terms of differing selectivity
- Compared the performance against filtering documents of interest using Spark

Benchmarking Streaming Computation Engines

University of Waterloo
Sep 2017 - Dec 2017     Github

- Evaluated current caching mechanism by examining hit ratio of cached RDD partitions
- Analyzed how different caching replacement policies affect the performance of Spark
- Implemented a report generator on cached RDD usage

Benchmarking Streaming Computation Engines

University of Waterloo
Sep 2017 - Dec 2017

- Analyzing latency and throughputs of Storm and Spark Streaming
- Benchmarking against TPCx-IoT spec and proposing proper use case for each engine

1 Announcement of Firefox Voice with wakeword detection