Résumé
Senior Principal Scientist, Adobe
I’m working on applying machine learning towards understanding the structure and semantics of documents. Most recently, I’ve been working on leveraging LLMs for document-related workflows.
Acrobat AI Assistant
These days, I’m working on the Acrobat AI Assistant. From Adobe blog:
- AI Assistant: AI Assistant recommends questions based on a PDF’s content and answers questions about what’s in the document — all through an intuitive conversational interface.
- Generative summary: Get a quick understanding of the content inside long documents with short overviews in easy-to-read formats.
- Intelligent citations: Adobe’s custom attribution engine and proprietary AI generate citations so customers can easily verify the source of AI Assistant’s answers.
- Easy navigation: Clickable links help customers quickly find what they need in long documents so they can focus their time exploring and actioning the most important information.
Press coverage: MIT Technology Review - Transforming document understanding and insights with generative AI, The Verge - Adobe Acrobat adds generative AI to ‘easily chat with documents’, Inc. - Adobe’s New AI Assistant Is Designed to Read PDFs for You, CNBC - Adobe launches AI assistant that can search and summarize PDFs.
Liquid Mode
Previously, I developed the ML model powering Acrobat Liquid Mode.
From Adobe blog:
With the push of a button, Liquid Mode automatically reformats text, images, and tables for quick navigation and consumption on small screens. Powered by Adobe Sensei, Liquid Mode uses AI and machine learning in the background to understand and identify parts of a PDF, like headings, paragraphs, images, lists, tables, and more. It also attempts to understand the hierarchy and ordering of those parts to reformat a static PDF into a more dynamic and customizable experience.
Press coverage: The Wall Street Journal - Adobe Freshens Up Its PDF for Mobile, Fortune - Adobe 2020: Change the World, Fast Company - Adobe finally figured out how to make PDFs make sense on a phone, CNET - Adobe peps up PDF on smartphones with AI-powered Liquid reformatting, Business Insider - Adobe exec explains why the firm is using AI to transform the way people access and use documents on mobile, TechCrunch - Adobe’s ‘Liquid Mode’ uses AI to automatically redesign PDFs for mobile devices, Axios - Adobe reinvents PDF to work better on phones, VentureBeat - Adobe’s Liquid Mode leverages AI to reformat PDFs for mobile devices, Engadget - Adobe’s Liquid Mode uses AI to make PDFs easier to read on phones.
TIME called Liquid Mode as one of their Best Inventions of 2023 in the software/apps category.
Acrobat and Reader Sandboxes
Before that, I worked in application security, and specifcally on the design and implementation of the Adobe Reader Protected Mode sandbox. ASSET has some posts anouncing and describing it. Which was later integrated into Acrobat Protected View Sandbox.
From ASSET:
Adobe Reader Protected Mode represents an exciting new advancement in attack mitigation. Even if an exploitable security vulnerability is found by an attacker, Adobe Reader Protected Mode will help prevent the attacker from writing files, changing registry keys or installing malware on potential victims’ computers.
Press coverage: PCWorld - Why a “Sandbox” Makes Adobe Reader More Secure, ComputerWeekly - Sandboxing for secure app development: Adobe Reader’s ‘protected mode’, ZDNet - Adobe adding ‘sandbox’ to PDF Reader to ward off hacker attacks, PCMag - Adobe Explains More of Reader X Protected Mode.
Research
Patents and publications
See my Google Scholar profile
- Issued patents:
- Heading identification and classification for a digital document (US Patent 10,956,731)
- Identifying artifacts in digital documents (US Patent 10,949,604)
- Document structure identification using post-processing error correction (US Patent 11,783,610)
- Explanatory visualizations for object detection (US Patent 11,227,159)
- Machine learning prediction and document rendering improvement based on content order (US Patent 11,508,173)
- Automatic semantic labeling of form fields with limited annotations (US Patent 11,880,648)
- Extracting document hierarchy using a multimodal, layer-wise link prediction neural network (US Patent 12,333,844)
- Publications:
Vulnerability Research – Bug Bounties and CVEs
- Listed in the Top 50 security researchers for 2015 by Microsoft Security Response Center (MSRC).
- Shared the $1000 bounty and mentioned in The Chromium Projects Security Hall of Fame for a reporting a Chrome sandbox security bug.
- Reported and credited in 18 CVEs for Windows and Internt Explorer sandbox vulnerabilities; see my CVE credits page.
Honors and Awards
- Received several reward checks from Don Knuth.
- Was selected by Don Knuth to proof-read his books Selected Papers on Design of Algorithms and Selected Papers on Fun and Games (as part of a 4-member team).
- Picked up several prizes at the CodeChef competetive programming contests
- Was among the top 20 students to qualify the Indian National Physics Olympiad (InPhO) in 1999
Programming Languages
- Recently, I’ve been mostly been using Python (with TensorFlow) for the ML projects
- My prior work was largely in C++ (on Windows and Mac), with heavy x86-assembly debugging.
- I have a reasonably good understanding of the PDF format
- For jun, I enjoy programming in EMACS LISP and Haskell
Education
B.E. (with honors) in Computer Science, BITS Pilani, India (August 1999 – June 2003). I recevied a Bronze Medal (CGPA 9.92/10) and many merit scholarships.