Nazmul Karim

I am a Research Scientist at Bosch, working on the foundation model for sensor and wi-fi data interpretation. I recently completed my Ph.D. journey at the LCWN Lab, UCF, where I was advised by Prof. Nazanin Rahnavard and co-advised by Prof. Mubarak Shah. I have a broad interest in various topics in computer vision and machine learning. My Ph.D. research primarily focused on safe, responsible, and robust AI; including adversarial attacks and defenses, out-of-distribution (OOD) robustness, domain adaptation, and learning with noisy labels. I have also worked on compressive sensing, 3D Scene generation, video generation, continual learning, and multi-modal learning. So far, I have published 8 first-authored top-tier conference/journal papers.

Email  /  CV  /  Google Scholar  /  Twitter  /  LinkedIn  /  GitHub

  July 2024: My work on AI security got accepted to ACM CCS 2024
  July 2024: My work on AI security got accepted to ECCV 2024
  July 2024: LatentEditor got accepted to ECCV 2024
  July 2024: Free-Editor got accepted to ECCV 2024
  February 2024: Joined Bosch as a Research Scientist
  December 2023: Joined UCF CRCV as a Postdoctoral Scientist
  November 2023: Successfully defended my PhD dissertation
  May 2023: Started summer internship at Amazon Web Services
  February 2023: My work on source free domain adaptation got accepted to CVPR 2023
  May 2022: Started summer internship at SRI International
  April 2022: Paper accepted to SPIE 2022
  February 2022: Three papers accepted to CVPR 2022
  October 2021: Paper accepted to IEEE Transaction on Forensics and Information Security (TIFS)
  May 2020: Completed MS in Computer Engineering
  June 2019: Paper accepted to IEEE MLSP 2019

We propose a novel training-free 3D scene editing technique, FREE-EDITOR, which allows users to edit 3D scenes without further re-training the model during test time. Our proposed method successfully avoids the multi- view style inconsistency issue in SOTA methods with the help of a “single-view editing” scheme. Specifically, we show that editing a particular 3D scene can be performed by only modifying a single view.

We introduce LATENTEDITOR, an innovative framework designed to empower users with the ability to perform precise and locally controlled editing of neural fields using text prompts. Leveraging denoising diffusion models, we successfully embed real-world scenes into the latent space, resulting in a faster and more adaptable NeRF backbone for editing compared to traditional methods.

Our analysis shows that such manipulation can make the backdoor model converge to a bad local minima, i.e., sharper minima as compared to a benign model. Intuitively, the backdoor can be purified by re-optimizing the model to smoother minima. However, a naïve adoption of any optimization targeting smoother minima can lead to sub-optimal purification techniques hampering the clean test accuracy. Hence, to effectively obtain such re-optimization, inspired by our novel perspective establishing the connection between backdoor removal and loss smoothness, we propose Fisher Information guided Purification (FIP), a novel backdoor purification framework. Proposed FIP consists of a couple of novel regularizers that aid the model in suppressing the backdoor effects and retaining the acquired knowledge of clean data distribution throughout the backdoor removal procedure by exploiting the knowledge of Fisher Information Matrix (FIM).

In this paper, we introduce Neural Mask Fine-Tuning (NFT), a method designed to optimally reorganize neuron activities to eliminate backdoor effects. NFT leverages simple data augmentation techniques such as MixUp, simplifying the trigger synthesis process and removing the need for adversarial search modules. Our findings indicate that direct weight fine-tuning with limited validation data leads to overfitting and poor post-purification clean test accuracy. To address this, NFT fine-tunes neural masks rather than model weights and incorporates a mask regularizer to minimize model drift during purification. NFT demonstrates high efficiency in both runtime and sample usage, capable of removing backdoors with just a single sample per class. We validate NFT’s effectiveness through extensive experiments across various tasks, including image classification, object detection, video action recognition, 3D point cloud processing, and natural language processing.

A curriculum learning-aided self-training framework for SFDA that adapts efficiently and reliably to changes across domains based on selective pseudo-labeling. Specifically, we employ a curriculum learning scheme to promote learning from a restricted amount of pseudo labels selected based on their reliabilities. This simple yet effective step successfully prevents label noise propagation during different stages of adaptation and eliminates the need for costly memory-bank based label refinement.

UNICON is a robust sample selection approach for training with high label noise. It incorporates a Jensen-Shannon divergence-based uniform sample selection mechanism and contrastive learning.

The task of continual learning requires careful design of algorithms that can tackle catastrophic forgetting. However, the noisy label, which is inevitable in a real-world scenario, seems to exacerbate the situation. While very few studies have addressed the issue of continual learning under noisy labels, long training time and complicated training schemes limit their applications in most cases. In contrast, we propose a simple purification technique to effectively cleanse the online data stream that is both cost-effective and more accurate.

We propose a simple yet effective generalized OOD detection method independent of out-of-distribution datasets. Our approach relies on self-supervised feature learning of the training samples, where the embeddings lie on a compact low-dimensional space. Motivated by the recent studies that show self-supervised adversarial contrastive learning helps robustify the model, we empirically show that a pre-trained model with self-supervised contrastive learning yields a better model for uni-dimensional feature learning in the latent space.

Deep neural networks (DNNs) designed for computer vision and natural language processing tasks cannot be directly applied to the radio frequency (RF) datasets. To address this challenge, we propose to convert the raw RF data to data types that are suitable for off-the-shelf DNNs by introducing a convolutional transform technique.

We propose a detector that is based on the analysis of the intrinsic DNN properties; that are affected due to the Trojaning process. For a comprehensive analysis, we develop Odysseus1 , the most diverse dataset to date with over 3,000 clean and Trojan models. Odysseus covers a large spectrum of attacks; generated by leveraging the versatility in trigger designs and source to target class mappings. Our analysis results show that Trojan attacks affect the classifier margin and shape of decision boundary around the manifold of clean data. Exploiting these two factors, we propose an efficient Trojan detector that operates without any knowledge of the attack and significantly outperforms existing methods.

A reinforcement-learning-based non-uniform compressed sensing (NCS) framework for time-varying signals is introduced. The proposed scheme, referred to as RL-NCS, aims to boost the performance of signal recovery through an optimal and adaptive distribution of sensing energy among two groups of coefficients of the signal, referred to as region of interest (ROI) coefficients and non-ROI coefficients. The coefficients in ROI usually have greater importance and need to be reconstructed with higher accuracy compared to non-ROI coefficients.