Senior Principal Scientist, Adobe

I’m working on applying machine learning towards understanding the structure and semantics of documents. Most recently, I’ve been working on leveraging LLMs for document-related workflows.

Acrobat AI Assistant

These days, I’m working on the Acrobat AI Assistant. From Adobe blog:

  • AI Assistant: AI Assistant recommends questions based on a PDF’s content and answers questions about what’s in the document — all through an intuitive conversational interface.
  • Generative summary: Get a quick understanding of the content inside long documents with short overviews in easy-to-read formats.
  • Intelligent citations: Adobe’s custom attribution engine and proprietary AI generate citations so customers can easily verify the source of AI Assistant’s answers.
  • Easy navigation: Clickable links help customers quickly find what they need in long documents so they can focus their time exploring and actioning the most important information.

Press coverage: MIT Technology Review - Transforming document understanding and insights with generative AI, The Verge - Adobe Acrobat adds generative AI to ‘easily chat with documents’, Inc. - Adobe’s New AI Assistant Is Designed to Read PDFs for You, CNBC - Adobe launches AI assistant that can search and summarize PDFs.

Liquid Mode

Previously, I developed the ML model powering Acrobat Liquid Mode.

From Adobe blog:

With the push of a button, Liquid Mode automatically reformats text, images, and tables for quick navigation and consumption on small screens. Powered by Adobe Sensei, Liquid Mode uses AI and machine learning in the background to understand and identify parts of a PDF, like headings, paragraphs, images, lists, tables, and more. It also attempts to understand the hierarchy and ordering of those parts to reformat a static PDF into a more dynamic and customizable experience.

Press coverage: The Wall Street Journal - Adobe Freshens Up Its PDF for Mobile, Fortune - Adobe 2020: Change the World, Fast Company - Adobe finally figured out how to make PDFs make sense on a phone, CNET - Adobe peps up PDF on smartphones with AI-powered Liquid reformatting, Business Insider - Adobe exec explains why the firm is using AI to transform the way people access and use documents on mobile, TechCrunch - Adobe’s ‘Liquid Mode’ uses AI to automatically redesign PDFs for mobile devices, Axios - Adobe reinvents PDF to work better on phones, VentureBeat - Adobe’s Liquid Mode leverages AI to reformat PDFs for mobile devices, Engadget - Adobe’s Liquid Mode uses AI to make PDFs easier to read on phones.

TIME called Liquid Mode as one of their Best Inventions of 2023 in the software/apps category.

Acrobat and Reader Sandboxes

Before that, I worked in application security, and specifcally on the design and implementation of the Adobe Reader Protected Mode sandbox. ASSET has some posts anouncing and describing it. Which was later integrated into Acrobat Protected View Sandbox.

From ASSET:

Adobe Reader Protected Mode represents an exciting new advancement in attack mitigation. Even if an exploitable security vulnerability is found by an attacker, Adobe Reader Protected Mode will help prevent the attacker from writing files, changing registry keys or installing malware on potential victims’ computers.

Press coverage: PCWorld - Why a “Sandbox” Makes Adobe Reader More Secure, ComputerWeekly - Sandboxing for secure app development: Adobe Reader’s ‘protected mode’, ZDNet - Adobe adding ‘sandbox’ to PDF Reader to ward off hacker attacks, PCMag - Adobe Explains More of Reader X Protected Mode.

Research

Patents and publications

See my Google Scholar profile

  • Issued patents:
    • Heading identification and classification for a digital document (US Patent 10,956,731)
    • Identifying artifacts in digital documents (US Patent 10,949,604)
    • Document structure identification using post-processing error correction (US Patent 11,783,610)
    • Explanatory visualizations for object detection (US Patent 11,227,159)
    • Machine learning prediction and document rendering improvement based on content order (US Patent 11,508,173)
    • Automatic semantic labeling of form fields with limited annotations (US Patent 11,880,648)
  • Publications:
    • Black-Box Explanation of Object Detectors via Saliency Maps (CVPR 2021, website)
    • LayerDoc: Layer-Wise Extraction of Spatial Hierarchical Structure in Visually-Rich Documents (WACV 2023)

Vulnerability Research – Bug Bounties and CVEs

Honors and Awards

Programming Languages

  • Recently, I’ve been mostly been using Python (with TensorFlow) for the ML projects
  • My prior work was largely in C++ (on Windows and Mac), with heavy x86-assembly debugging.
  • I have a reasonably good understanding of the PDF format
  • For jun, I enjoy programming in EMACS LISP and Haskell

Education

B.E. (with honors) in Computer Science, BITS Pilani, India (August 1999 – June 2003). I recevied a Bronze Medal (CGPA 9.92/10) and many merit scholarships.