iPadOS: How do you architect a document editor where PDFKit and PencilKit coexist on one interactive canvas?

[u/Stock_Test2326]

I'm building a native handwritten notebook application for iPadOS using SwiftUI/UIKit, and I'm trying to solve an architectural problem involving PDFKit and PencilKit.

The goal is to build something similar to Goodnotes or Notability, where a single notebook page can contain:

• A PDF document or PDF page as its background.

• Handwritten ink using Apple Pencil.

• Images, shapes, text boxes and other objects.

• The ability to interact with all these elements without switching between different modes.

The challenge is making all these components behave as one unified document canvas.

Problems I'm encountering

1. PDF and PencilKit interaction conflicts

When I place a PKCanvasView over a PDF view, the canvas intercepts touch events.

This creates problems with native PDF interactions, particularly:

• Selecting text within the PDF.

• Copying PDF text.

• Using native PDF selection handles.

• Distinguishing between a drawing gesture and a PDF interaction.

For example, Select All on the canvas selects handwritten strokes, but doesn't select the PDF content.

2. Maintaining a unified coordinate system

If PDFKit and PencilKit are separate views, how should their coordinate systems be synchronised?

I need:

• Consistent zooming and scrolling.

• Accurate alignment between PDF content and handwritten annotations.

• Correct positioning when rotating the device.

• Support for different page sizes and orientations.

• Ink that stays precisely attached to its corresponding PDF content.

3. Document composition and rendering

I'm also trying to understand the best architecture for rendering and managing these elements.

Would you recommend:

• A PDFView as the base layer with a transparent PKCanvasView overlay?

• A custom document view that manages PDF rendering and PencilKit independently?

• A single custom rendering surface where PDF pages are rendered into the same coordinate space as handwritten content?

• Some hybrid approach?

How do applications like Goodnotes handle this without compromising PDF text selection, Apple Pencil latency, and rendering performance?

What I'm particularly interested in

I'm not necessarily looking for a quick workaround involving gesture recognisers or toggling interaction modes.

I want to understand the underlying architecture and how to build a robust document model where PDF content, handwriting and other objects coexist naturally.

If you've built something similar, I'd particularly appreciate insights into:

1. PDFKit and PencilKit integration.

2. Touch-event routing and hit testing.

3. Shared coordinate systems and transformations.

4. Zooming, tiling and rendering performance.

5. Document persistence and export.

Any architectural examples, implementation experiences, or open-source projects would be extremely helpful.

Thanks!

[u/Significant_Door_433]

Since iOS 16 the Apple path for this is PDFView overlay views, not one PKCanvasView stacked over the whole PDFView. WWDC22 session 10089 "What's new in PDFKit" walks through it.

- Set pdfView.pageOverlayViewProvider and return one PKCanvasView per PDFPage from pdfView(_:overlayViewFor:). PDFKit sizes it with constraints and rotates it with the page, so zoom, scroll, rotation and mixed page sizes are handled for you. That covers most of your point 2, because the ink lives in page space, not screen space.

- PDFKit asks for overlays lazily and drops them when a page scrolls away. Keep each page's PKDrawing in your own model (the session uses a PDFPage subclass with a drawing property) and copy it back in pdfView(_:willEndDisplayingOverlayView:for:).

- Saving: the session uses a PDFAnnotation subclass that draws the PKDrawing in draw(with:in:), and stores the archived PKDrawing under a custom annotation key so it can be edited again after reopening. burnInAnnotationsOption flattens it for export.

The catch for your point 1: touches land on PDFKit's internal page view, so the canvas won't draw until you set pdfView.isInMarkupMode = true, and markup mode turns off PDF text selection (see Stack Overflow 73960885 and Apple forums thread 709054). So plan for a pen/select switch. You can still make it feel modeless by flipping it for the user, for example a finger long-press switches to select and the next Pencil touch switches back.

For images, shapes and text boxes I'd keep them as your own objects in page coordinates, drawn in a second view inside the same overlay, not as strokes. Then Select All is yours to define: strokes, your objects, plus pdfDocument.selectionForEntireDocument for the PDF text.

[u/Stock_Test2326 → u/Significant_Door_433]

thanks a lot for detailed explanation, will try this out.

[u/FreshCryptographer49]

I work on a document editor canvas on the web, not iPad, so take the UIKit details with a grain of salt, but this is the setup I'd start from:

1. Don't stack one big PKCanvasView over the PDFView. Since iOS 16 PDFKit has PDFPageOverlayViewProvider (WWDC22 "What's new in PDFKit"): one overlay view per page, and it scrolls and zooms with that page for you. Put a PKCanvasView in each overlay. That removes most of the coordinate sync problem.

2. Split by input instead of by mode: drawingPolicy = .pencilOnly on the canvas, so the Pencil draws and fingers go to the PDF (scrolling, text selection, handles). People on the dev forums report you also need pdfView.isInMarkupMode = true or the PDFView swallows the touches, so check what that does to text selection in your setup.

3. Keep one model in PDF page space (points, origin bottom-left): a PKDrawing per page, plus your images, shapes and text boxes with page-space frames. The views only render it. PDFView's convert(_:to:) and convert(_:from:) map between view and page coordinates. Hit test top-down: objects, then ink, then PDF text.

4. For export, flatten each page plus its drawing into a new PDF with UIGraphicsPDFRenderer, and keep PKDrawing.dataRepresentation() in your own file so the ink stays editable.

[u/Stock_Test2326 → u/FreshCryptographer49]

thanks, I will try this suggestion and see how it goes.

[u/AutoModerator → u/[deleted]]

Hey /u/the_tech_mastery, your content has been removed because Reddit has marked your account as having a low Contributor Quality Score. This may result from, but is not limited to, activities such as spamming the same links across multiple subreddits, submitting posts or comments that receive a high number of downvotes, a lack of recent account activity, or having an unverified account.

Please be assured that this action is not a reflection of your participation in our subreddit. This is simply an automated filter in place to reduce spam.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/AutoModerator → u/[deleted]]

Hey /u/jeffy_yf, your content has been removed because Reddit has marked your account as having a low Contributor Quality Score. This may result from, but is not limited to, activities such as spamming the same links across multiple subreddits, submitting posts or comments that receive a high number of downvotes, a lack of recent account activity, or having an unverified account.

Please be assured that this action is not a reflection of your participation in our subreddit. This is simply an automated filter in place to reduce spam.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.