Some Research Topics of My Current Interests

Integration of Deep Learning and First-Order Logic (Knowledge)

Towards developing an end-to-end learning framework to integrate high-level logic representation and deep learning architectures. Our representative works (with applications to NLP tasks) by far:

Deep Inductive Logic Reasoning for Multi-Hop Reading Comprehension.
Wenya Wang and Sinno Jialin Pan.
To appear in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL-22). Dublin. May 22-27, 2022. (Full paper)

Variational Deep Logic Network for Joint Inference of Entities and Relations. [Bibtex]
Wenya Wang and Sinno Jialin Pan.
In Computational Linguistics (CL). Volume 47, NO. 4, Pages 775-812, 2021.

Deep Weighted MaxSAT for Aspect-based Opinion Extraction.
Meixi Wu, Wenya Wang, and Sinno Jialin Pan.
In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP-20).
[Code]

Integrating Deep Learning with Logic Fusion for Information Extraction.
Wenya Wang and Sinno Jialin Pan.
In Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI-20).
[Code]

Our previous attempts on encoding knowledge in terms of syntactic information for NLP tasks (fine-grained sentiment analysis):

Syntactically-Meaningful and Transferable Recursive Neural Networks for Aspect and Opinion Extraction. [Bibtex]
Wenya Wang and Sinno Jialin Pan.
In Computational Linguistics (CL). Volume 45, Issue 4, Pages 705-736, 2019.

Memory Networks for Fine-grained Opinion Mining. [Bibtex]
Wenya Wang, Sinno Jialin Pan and Daniel Dahlmeier.
In Artificial Intelligence (AIJ). Volume 265, Pages 1-17, 2018.

Transferable Interactive Memory Network for Domain Adaptation in Fine-grained Opinion Extraction. [Bibtex]
Wenya Wang and Sinno Jialin Pan.
In Proceedings of the 33rd AAAI Conference on Artificial Intelligence (AAAI-19).

Recursive Neural Structural Correspondence Network for Cross-domain Aspect and Opinion Co-Extraction. [Bibtex]
Wenya Wang and Sinno Jialin Pan.
In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL-18).
[Code]

Transition-based Adversarial Network for Cross-lingual Aspect Extraction. [Bibtex]
Wenya Wang and Sinno Jialin Pan.
In Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI-18).

Coupled Multi-layer Attentions for Co-extraction of Aspect and Opinion Terms. [Bibtex]
Wenya Wang, Sinno Jialin Pan, Daniel Dahlmeier and Xiaokui Xiao.
In Proceedings of the 31st AAAI Conference on Artificial Intelligence (AAAI-17).
[Code & Dataset]

Recursive Neural Conditional Random Fields for Aspect-based Sentiment Analysis. [Bibtex]
Wenya Wang, Sinno Jialin Pan, Daniel Dahlmeier and Xiaokui Xiao.
In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP-16).
[Code & Dataset]

Deep Models Compression

Towards learning compressed deep models while preserving satisfactory prediction performance for edge or resource-limted devices. Our representative works by far:

Storage Efficient and Dynamic Flexible Runtime Channel Pruning via Deep Reinforcement Learning.
Jianda Chen, Shangyu Chen, and Sinno Jialin Pan.
In Advances in Neural Information Processing Systems 33 (NeurIPS-20).
[Code]

MetaQuant: Learning to Quantize by Learning to Penetrate Non-differentiable Quantization.
Shangyu Chen, Wenya Wang and Sinno Jialin Pan.
In Advances in Neural Information Processing Systems 32 (NeurIPS-19).
[Code]

Cooperative Pruning in Cross-Domain Deep Neural Network Compression.
Shangyu Chen, Wenya Wang and Sinno Jialin Pan.
In Proceedings of the 28th International Joint Conference on Artificial Intelligence (IJCAI-19).
[Code]

Deep Neural Network Quantization via Layer-Wise Optimization using Limited Training Data.
Shangyu Chen, Wenya Wang and Sinno Jialin Pan.
In Proceedings of the 33rd AAAI Conference on Artificial Intelligence (AAAI-19).
[Code]

Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeon.
Xin Dong, Shangyu Chen and Sinno Jialin Pan.
In Advances in Neural Information Processing Systems 30 (NIPS-17).
[Code]

Distributed (or Federated) Multi-task Learning

Towards developing communication-efficient distributed learning frameworks for multi-task learning (i.e, jointly and efficiently learning multiple tasks in a geo-distributed computing environment). Our representative works by far:

Communication-Efficient Distributed Multi-Task Learning with Matrix Sparsity Regularization.
Qiang Zhou, Yu Chen and Sinno Jialin Pan.
In Machine Learning (MLJ). Volume 109, No. 3, Pages 569-601, 2020.

Distributed Multi-task Relationship Learning.
Sulin Liu, Sinno Jialin Pan and Qirong Ho
In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-17).

Other related works on distributed learning:

Communication-Efficient Distributed PCA by Riemannian Optimization.
Long-Kai Huang and Sinno Jialin Pan.
In Proceedings of the 37th International Conference on Machine Learning (ICML-20).

Communication-Efficient Distributed Primal-Dual Algorithm for Saddle Point Problems.
Yaodong Yu*, Sulin Liu* and Sinno Jialin Pan.
In Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence (UAI-17).
(Note: * indicates equal contributions.)

Learning from Time Series Data

Towards developing a general learning framework for time series data, where statistical and temporal patterns underlying time series data can be effectively captured. Our representative works (with an application to sensor-based activity recognition) by far:

Weakly-Supervised Sensor-based Activity Segmentation and Recognition via Learning from Distributions.
Hangwei Qian, Sinno Jialin Pan, and Chunyan Miao.
In Artificial Intelligence (AIJ). Volume 292, 2021.
[Code]

Latent Independent Excitation for Generalizable Sensor-based Cross-Person Activity Recognition.
Hangwei Qian, Sinno Jialin Pan, and Chunyan Miao.
In Proceedings of the 35th AAAI Conference on Artificial Intelligence (AAAI-21).
[Code]

A Novel Distribution-Embedded Neural Network for Sensor-Based Activity Recognition.
Hangwei Qian, Sinno Jialin Pan, Bingshui Da and Chunyan Miao.
In Proceedings of the 28th International Joint Conference on Artificial Intelligence (IJCAI-19).
[Code]

Distribution-based Semi-Supervised Learning for Activity Recognition.
Hangwei Qian, Sinno Jialin Pan and Chunyan Miao.
In Proceedings of the 33rd AAAI Conference on Artificial Intelligence (AAAI-19).
[Code]

Sensor-based Activity Recognition via Learning from Distributions.
Hangwei Qian, Sinno Jialin Pan and Chunyan Miao.
In Proceedings of the 32nd AAAI Conference on Artificial Intelligence (AAAI-18).
[Code]