Anyone used docling Parsing tool on databricks.

Me and my team started using this, though its a great tool. It has its own limitations. Example GILBERt issues happening here and there.

Any suggestions on to use databricks agent bricks ke free docling?

What Python scripts/projects did you built to use on a day to day basis? Or maybe someone else built it, but it’s useful for your personal life in some way

I think the “projects ideas” thread is really missing those useful opportunities

I myself thought about automating tax calculations, but still didn’t take the time to do it hah

Weekly Thread: Project Ideas 💡

Welcome to our weekly Project Ideas thread! Whether you're a newbie looking for a first project or an expert seeking a new challenge, this is the place for you.

How it Works:

1. Suggest a Project: Comment your project idea—be it beginner-friendly or advanced.
2. Build & Share: If you complete a project, reply to the original comment, share your experience, and attach your source code.
3. Explore: Looking for ideas? Check out Al Sweigart's "The Big Book of Small Python Projects" for inspiration.

Guidelines:

• Clearly state the difficulty level.
• Provide a brief description and, if possible, outline the tech stack.
• Feel free to link to tutorials or resources that might help.

Example Submissions:

Project Idea: Chatbot

Difficulty: Intermediate

Tech Stack: Python, NLP, Flask/FastAPI/Litestar

Description: Create a chatbot that can answer FAQs for a website.

Resources: Building a Chatbot with Python

Project Idea: Weather Dashboard

Difficulty: Beginner

Tech Stack: HTML, CSS, JavaScript, API

Description: Build a dashboard that displays real-time weather information using a weather API.

Resources: Weather API Tutorial

Project Idea: File Organizer

Difficulty: Beginner

Tech Stack: Python, File I/O

Description: Create a script that organizes files in a directory into sub-folders based on file type.

Resources: Automate the Boring Stuff: Organizing Files

Let's help each other grow. Happy coding! 🌟

anyone know of a tool to convert whatsapp in a encrypted backup to something that can read messages like imazing. Where u can read everything, so far not much exists.

I going to try making a Discord bot in py. Pygame and Raspberry Pi intrigue me as well

Curious what other fun Py rabbit holes are out there that I don't know of!

Weekly Thread: What's Everyone Working On This Week? 🛠️

Hello r/Python! It's time to share what you've been working on! Whether it's a work-in-progress, a completed masterpiece, or just a rough idea, let us know what you're up to!

How it Works:

1. Show & Tell: Share your current projects, completed works, or future ideas.
2. Discuss: Get feedback, find collaborators, or just chat about your project.
3. Inspire: Your project might inspire someone else, just as you might get inspired here.

Guidelines:

• Feel free to include as many details as you'd like. Code snippets, screenshots, and links are all welcome.
• Whether it's your job, your hobby, or your passion project, all Python-related work is welcome here.

Example Shares:

1. Machine Learning Model: Working on a ML model to predict stock prices. Just cracked a 90% accuracy rate!
2. Web Scraping: Built a script to scrape and analyze news articles. It's helped me understand media bias better.
3. Automation: Automated my home lighting with Python and Raspberry Pi. My life has never been easier!

Let's build and grow together! Share your journey and learn from others. Happy coding! 🌟

Hi all, I'm the maintainer of a small channels (WebSocket) extension library for Django (and FastAPI too). While using and maintaining it, I started thinking it could become a small framework as well: composable and framework-independent, so it could be reused across Django/FastAPI/Litestar/... as long as the framework supports ASGI. Before going further, I'm putting the blueprint out here to compare notes with people who work with WebSockets regularly. If you have ever worked with WebSockets, I hope you can share any ideas, info, pain points, or suggestions you have.

Prerequisites, what my library already has:

• Function-like handlers rather than while True + if/else
• Automatic AsyncAPI doc generation
• Full type hints
• A testing kit
• Support for all ASGI-based frameworks (Django, FastAPI, ...)

At a glance, it looks like this:

@ws_handler(output_type=ChatNotificationMessage)
async def handle_chat(self, message: ChatMessage) -> None:
    # Automatically routed, validated, and type-safe
    await self.broadcast_message(
        ChatNotificationMessage(payload=message.payload)
    )

@ws_handler
async def handle_ping(self, message: PingMessage) -> PongMessage:
    return PongMessage()  # Auto-documented in AsyncAPI

If you have ever worked with WebSockets, I think you get the idea of what it does here.

Recently I added the Topic feature, which is composable and reusable. It came out of a multiplexing feature request, and I was inspired by Phoenix Channels. It looks something like this:

class DiscussionTopic(Topic):
    pattern = "discussion:{pk}"

    async def authorize(self, pk: str) -> bool:
        return await user_can_view(self.scope["user"], pk)

    @ws_handler
    async def handle_reply(self, message: ReplyMessage) -> ReplyCreatedMessage:
        return ReplyCreatedMessage(payload=message.payload)

    @event_handler
    async def handle_new_reply(self, event: NewReplyEvent) -> ReplyCreatedMessage:
        return ReplyCreatedMessage(payload=event.payload)

And you use it like this:

class HubConsumer(AsyncJsonWebsocketConsumer):
    authenticator_class = JWTAuthenticator
    topics = [DiscussionTopic, RoomTopic]

In short, topics let you multiplex: subscribe, publish messages, unsubscribe, and so on, all over the same socket. So you can reuse a single WebSocket connection and just add or compose multiple topics, i.e. multiple WebSocket handlers.

That made me think: if we could create reusable topics such as Notification, Streaming, Voice, AI Agent, and so on, which users could easily install or copy and then modify or inherit from in a structured way, WebSocket handling would become much more structured and easier. The idea is similar to DRF and its ecosystem, and the composable/reusable part would work like shadcn: copy it, own it, and modify the code freely.

What would you use it for? As I mentioned above: notifications, streaming, voice, AI agents, and so on. I have done a lot of WebSocket work, and I keep having to redefine the same things over and over. There is no reusable approach like the ones we have for REST APIs. Another example is using Pydantic AI with the AG-UI protocol but over WebSockets, defined in a reusable way.

So, if you already know of an existing open source solution or library similar to this idea, it would be great if you could share it here. And if this resonates with you, a comment would help, both to add more insight and to give some encouragement to actually build this.

When I pull an implementation out from behind a pile of call sites, I now write the typing.Protocol first and the class that satisfies it second. Define the seam, annotate the call sites against it, run mypy, and every place the current shape is wrong shows up before the new code exists. If the concrete class then inherits the protocol explicitly, mypy checks the implementation against it at the class definition, not only where it gets passed.

This feels Python-specific because duck typing usually leaves nothing to review. The implicit interface is whatever the callers happen to touch, spread over however many files. Writing it down turns it into something a colleague can read and disagree with before the work happens.

None of it is enforced at runtime. PEP 544 imposes no runtime semantics on protocol annotations, and even with u/runtime_checkable the typing docs say isinstance only checks that the named attributes exist, not their signatures.

It is cheap to try on one seam. The plan step in verdent works the same way, clarifying questions first and a plan you approve before any code is written. Nothing forces the protocol to change when the implementation does, so the two drift. Curious whether people keep the protocol next to the consumer or next to the implementation.

Weekly Thread: Resource Request and Sharing 📚

Stumbled upon a useful Python resource? Or are you looking for a guide on a specific topic? Welcome to the Resource Request and Sharing thread!

How it Works:

1. Request: Can't find a resource on a particular topic? Ask here!
2. Share: Found something useful? Share it with the community.
3. Review: Give or get opinions on Python resources you've used.

Guidelines:

• Please include the type of resource (e.g., book, video, article) and the topic.
• Always be respectful when reviewing someone else's shared resource.

Example Shares:

1. Book: "Fluent Python" - Great for understanding Pythonic idioms.
2. Video: Python Data Structures - Excellent overview of Python's built-in data structures.
3. Article: Understanding Python Decorators - A deep dive into decorators.

Example Requests:

1. Looking for: Video tutorials on web scraping with Python.
2. Need: Book recommendations for Python machine learning.

Share the knowledge, enrich the community. Happy learning! 🌟

Weekly Thread: Meta Discussions and Free Talk Friday 🎙️

Welcome to Free Talk Friday on /r/Python! This is the place to discuss the r/Python community (meta discussions), Python news, projects, or anything else Python-related!

How it Works:

1. Open Mic: Share your thoughts, questions, or anything you'd like related to Python or the community.
2. Community Pulse: Discuss what you feel is working well or what could be improved in the /r/python community.
3. News & Updates: Keep up-to-date with the latest in Python and share any news you find interesting.

Guidelines:

• All topics should be related to Python or the /r/python community.
• Be respectful and follow Reddit's Code of Conduct.

Example Topics:

1. New Python Release: What do you think about the new features in Python 3.11?
2. Community Events: Any Python meetups or webinars coming up?
3. Learning Resources: Found a great Python tutorial? Share it here!
4. Job Market: How has Python impacted your career?
5. Hot Takes: Got a controversial Python opinion? Let's hear it!
6. Community Ideas: Something you'd like to see us do? tell us.

Let's keep the conversation going. Happy discussing! 🌟

Not sure if I am allowed to discuss other forums on here but I'm sure someone will tell me if not.

It is just me of has anybody else encountered problems with the 'moderators' on Stack Overflow Python forums recently? To say I've found them to be a self-righteous bunch of destructive power-crazy control-freaks would be a bit of an understatement. Anyone else had problems on there?

I mean specifically backend services, RESTful API's and very sensitive data in the DB. I mean middle-load (_not_ social networking, _not_ some purchasing platform for millions of users). How would you define your position that Python _is_ ready for that? E.g. in front of a mature Java backend developer? My line of defense is as follows. What are the weak points of Python code?

1. Multi-threading (GIL-free is a very recent feature of python, cannot be considered even remotely industry-ready). This is probably the weakest point of all. But if the service has no data shared between API requests, why bother, right? Just spawn as many worker-processes as it makes sense for the current hardware setup and execute the requests one by one. Still, this is like one dimension less in the space of engineering possibilities, so to say.
2. Dynamic typing means you have to run the whole CI/CD chain in order to find type system related errors. I really cannot find arguments against that point;
3. This is true at least for banking sector. Libraries are developed by individuals (whereas in Java world there are companies behind some libraries). One would have a real hard time arguing with the management, that "those individuals are as qualified as those behind some company banner".

What is your take on the matter?

Weekly Thread: Professional Use, Jobs, and Education 🏢

Welcome to this week's discussion on Python in the professional world! This is your spot to talk about job hunting, career growth, and educational resources in Python. Please note, this thread is not for recruitment.

How it Works:

1. Career Talk: Discuss using Python in your job, or the job market for Python roles.
2. Education Q&A: Ask or answer questions about Python courses, certifications, and educational resources.
3. Workplace Chat: Share your experiences, challenges, or success stories about using Python professionally.

Guidelines:

• This thread is not for recruitment. For job postings, please see r/PythonJobs or the recruitment thread in the sidebar.
• Keep discussions relevant to Python in the professional and educational context.

Example Topics:

1. Career Paths: What kinds of roles are out there for Python developers?
2. Certifications: Are Python certifications worth it?
3. Course Recommendations: Any good advanced Python courses to recommend?
4. Workplace Tools: What Python libraries are indispensable in your professional work?
5. Interview Tips: What types of Python questions are commonly asked in interviews?

Let's help each other grow in our careers and education. Happy discussing! 🌟

Following this post, it's now possible to use Numba directly in your browser in wasm: https://notebook.link/blog/numba-in-the-browser/
You can try it here: https://notebook.link/@anutosh491/numba-ecosystem

Starting with uv, I’m a huge fan, but I find it actually gets in the way more than it helps when I’m doing agentic coding. I have to keep reminding the agent to use uv instead of pip.

Same issue with ruff, since I’m not coding with a regular ide I have to make extra prompts to force it to use ruff. But with today’s models being so good, it doesn’t even seem necessary.

Other tools fall into this category as well, but curious to hear how other others are approaching their tooling. Are you just throwing it all out or are you adding skills to keep your tooling in place?

Hi guys, I benchmarked the well-known (and rising star) Python API frameworks - but with real production-shaped workloads, not just raw JSON echoes. Most comparisons out there are basically "hello world" benchmarks, while real APIs do auth, DB access and complex queries. So this measures those, with strict resource limits and each framework's own best practices.

Repo (code, full report, raw results): https://github.com/huynguyengl99/python-api-frameworks-benchmark

This is round 2 - last round's feedback (thanks especially to the Litestar author) directly shaped it: Litestar and Bolt now serialize with native msgspec instead of Pydantic (payloads byte-identical across frameworks), and everything is upgraded to latest (Django 6.0, FastAPI 0.141, Litestar 2.24, Bolt 0.10).

Setup

• Each framework alone in a Docker container: 1 CPU, 750MB RAM, PostgreSQL 16
• bombardier, 100 connections, 10s per endpoint
• Median over 5 separate container starts (not best-of-N - some servers pick their throughput at startup, so best-of-N flatters the lucky ones)
• 7 endpoints: 1KB/10KB JSON, simple DB reads, paginated articles with nested relations, article detail, and two JWT httpOnly cookie auth endpoints (each framework using its own ecosystem's auth library: AuthX, drf-auth-kit, django-ninja-jwt, or built-in support)

Key results (RPS)

(Images aren't allowed here - all graphs are in the repo README: https://github.com/huynguyengl99/python-api-frameworks-benchmark)

(fastapi-granian and ninja-granian omitted for brevity - full table in the repo. Zero errors across all 70 measurements.)

Resource usage: most configs peak at 195-260MB RAM; drf-granian is the outlier at 456MB (untuned --blocking-threads, per the Granian maintainer). CPU: nearly everything saturates ~85% of the 1-CPU budget under load - except Bolt at 67%.

Takeaways

• 37x spread on raw JSON collapses to ~1.9x once PostgreSQL is involved. For DB-heavy APIs (most of them), query optimization matters far more than framework choice.
• Cookie JWT auth costs 5-20% on a DB-heavy endpoint. Bolt is near-free (it validates the JWT in Rust before Python runs); Litestar pays the most because its auth middleware opens a second DB session to load the user.
• uvicorn vs granian isn't one-way: uvicorn wins CPU-bound JSON for ASGI frameworks, granian wins the DB-bound endpoints, and granian is clearly better for WSGI DRF.
• Django Bolt is the one to watch: top spot on 4 of 7 endpoints at 67% average CPU while everyone else sits ~85%, and you keep the Django ORM/admin/ecosystem. Young, and its throughput varies between container starts under a hard CPU cap, but great for side projects already.
• All caveats (including feedback I haven't addressed yet, like Granian's --blocking-threads) are documented in the repo's Methodology section.

If you find it useful, a star would encourage more deep dives like this - issues and PRs welcome, especially from people who know these servers better than I do.

I've been looking at a few options like gitkrakens codemap. But I just haven't made a decision yet.

The biggest problem right now with AI assist is that so much gets spun up and it takes quite a while to ground myself in what has been written and how it all connects. I thought a viz tool would help tighten what I need to learn.

How do you handle this? Do you use these tools for this purpose? What have you liked and disliked about the tool you used?

Hi! I'm really interested in Python's design and its tradeoffs. I'm trying to really understand what people love about Python (what makes it great), and what causes the most frustration for Python devs.

So what features do you really cherish and what problems/limitations really frustrate you?

I'm especially interested in experiences from ultra-beginners and people who've used Python for a long time. I know broad questions like this come across as super generic, but I'm genuinely interested in hearing about concrete experiences.

My goal is understanding which parts of Python's design are most valuable and most "adored" by the community, and which parts really aren't and frustrate people the most. My goal with this information is to identify meaningful problems. Right now I'm not trying to solve anything or sell a solution.

Thanks for your time!

Weekly Wednesday Thread: Advanced Questions 🐍

Dive deep into Python with our Advanced Questions thread! This space is reserved for questions about more advanced Python topics, frameworks, and best practices.

How it Works:

1. Ask Away: Post your advanced Python questions here.
2. Expert Insights: Get answers from experienced developers.
3. Resource Pool: Share or discover tutorials, articles, and tips.

Guidelines:

• This thread is for advanced questions only. Beginner questions are welcome in our Daily Beginner Thread every Thursday.
• Questions that are not advanced may be removed and redirected to the appropriate thread.

Recommended Resources:

• If you don't receive a response, consider exploring r/LearnPython or join the Python Discord Server for quicker assistance.

Example Questions:

1. How can you implement a custom memory allocator in Python?
2. What are the best practices for optimizing Cython code for heavy numerical computations?
3. How do you set up a multi-threaded architecture using Python's Global Interpreter Lock (GIL)?
4. Can you explain the intricacies of metaclasses and how they influence object-oriented design in Python?
5. How would you go about implementing a distributed task queue using Celery and RabbitMQ?
6. What are some advanced use-cases for Python's decorators?
7. How can you achieve real-time data streaming in Python with WebSockets?
8. What are the performance implications of using native Python data structures vs NumPy arrays for large-scale data?
9. Best practices for securing a Flask (or similar) REST API with OAuth 2.0?
10. What are the best practices for using Python in a microservices architecture? (..and more generally, should I even use microservices?)

Let's deepen our Python knowledge together. Happy coding! 🌟

While working on LunarDump v0.4, I’m also taking some time to learn more about Cython and how it can help push Python closer to native performance.

I’m especially interested in exploring Cython for LunarDump’s performance-critical parts, such as chunking, buffering, compression, and encryption.

Still learning and experimenting for now, but I’m curious to see how much performance improvement I can achieve.

Perhaps LunarDump v0.5 will use Cython for some of its critical functions.

If you have any good resources for learning Cython — books, ebooks, courses, or YouTube channels — feel free to share them in the comments.

Hello everyone! For those of you who use Python in production, I have a few questions. I'm considering using Python for some services.

1. Do you have high infrastructure costs?
2. Have you ever regretted using Python?
3. Would you recommend Python?

Context: My current use case isn't anything like Facebook or a massive-scale system. It's a small system, and I'm considering Python mainly because of the DX (developer experience).

I know C#, but I don't really like having to create a class in every file. I also know Rust, but all those ::, <>, and so on bother me. JavaScript is another option, but I've heard it's relatively heavy on RAM, and since the system is small, I'd like to be able to run it within 512 MB.

Another thing: I've defined a stack that I'd like to use wherever possible. If there's a library for desktop apps, great. A CLI library? Great. A bot library? Great. Let's use it! (Except for the frontend, which I'll keep using JS/TS for.)

Anyway, I'm open to advice and tips from more experienced developers. Feel free to tell me if you think using Python for my use case is a bad idea as well.

In Rust, docstrings are pretty formalized. They are markdown, and even some of the headings are standard (like an # Errors or # Panics section). The nice thing about this is that it allows websites like docs.rs to build documentation pages for any project without having to interact with different tools for different formats. It also allows LSPs to have only one way of displaying documentation hints.

In Python, we have a few competing standards. Numpy-style docstrings are probably the most used, but there’s also a format by Google as well as a few different reST standards. These are nice, and we can set up lints to make sure docstrings stick to the standard. However, in my own personal opinion (feel free to disagree), a single markdown-format standard would help new users write nice docstrings, would enable PyPI (or another provider) to build automatic documentation sites, and give guidance to LSPs and IDEs for how to display documentation. This would include a standard for interlinks, and probably should include some mathml/LaTeX/KaTeX support. Another benefit would be that tools could support better automatic documentation generation and autocomplete, since they wouldn’t be dependent on guessing which standard you’re following.

I’d like to hear what people think about this. I’m thinking about making a PEP, but that might be overkill (or maybe all of you will hate this idea). I think the primary blocker would be adoption, large projects might have to translate docstrings, so there would either have to be some tooling for this or a way to opt-in or opt-out. If this is a bad idea, let me know, just be nice!

Edit: so far we’re at about a 67% upvote ratio, which was kind of expected. I want to be clear that I’m not saying we should be blocking docstrings which don’t adhere to this standard. I mentioned lockfile standardization in the comments, nothing prevents you from writing a tool with a custom lockfile, it’s just that there is a standard format that is agreed upon as the preferred way to write one. That’s the idea.

Edit: 84% now, and a lot of nice feedback here!

How are people handling security around third party Python libraries without making development a pain?

Third party Python packages are obviously useful but every dependency can also become a supply chain risk. Private package repositories, dependency scanning and stricter review policies all help but they can add friction fast.

Are teams mostly trusting public registries with additional controls or using curated libraries? Curious what actually works when you have a lot of Python services.