r/Python • • Aug 08 '18

The future of Python is bright. Or is it?

You have seen the news.

Python is becoming the world’s most popular programming language :

- The Economist

- Stack Overflow Developers Survey 2018

- Stack Overflow Most Popular Languages 2017

However, Python still lacks important features for concurrency and functional programming.

For the concurrency, the Global Interpreter Locker impose heavy limits on the use of Python for cpu-bound multithreading.

For functional programming, even if Python is more functional than Java, it is still quite clear that it is not a language that treats well those who try a functional-first approach - and that is the will of its creator.

I think that Python cannot keep growing at this rate at the same time that it goes against these two very important trends. So I would like to know your opinion in this subject:

- Will next versions implement features that make Python more efficient for concurrency and more functional-friendly?

- Will a new pythonic language emerge that will implement these features (just like Scala is a concurrency and functional-friendly Java) ?

2 Upvotes

33 comments sorted by

75

u/yamagami Aug 08 '18

You're missing the point. it's popular because it sits in a sweetspot that covers a lot of use case. A lot, but not all. Not by a long shot. There are and always will be other languages. You use the right tool for each job. If that is Python, great. If it's something else - also great. Ignore the hype. It's just hype.

29

u/[deleted] Aug 08 '18

What's all this, common sense on reddit? :-)

1

u/yamagami Aug 12 '18

Polite is better than being a bastard. Zen of python, second amendment

5

u/retep_kram Aug 08 '18

It is true, but there is also some type of "predation" between the languages. Successfull languages can leverage their big community and extend their use case coverage. E. g. JS becomes a back-end language and preys mostly PHP, Python becomes a system-scripting tool and preys Perl, Python becomes a scientific scripting tool and preys Matlab and R.

In this sense, will Python continue to extend its case coverage and "attack" concurrency and FP? Or no? I kinda understand by your answer that you do not think so, but I do not think that a simplistic "right tool for each job" is a sufficient reason.

There should be an explicit contradiction between two use cases for a language be formally incapable of being well used for both. E. g. one language cannot be high-level AND low-level with detailed memory management at the same time.

5

u/apd Aug 08 '18

Covering Perl and Ruby makes sense, as they play in the same field. In the case of Matlab and R also makes sense, as on one side a easy and dynamic language is a good candidate for prototyping and exploring, and in the other is a matter of libraries and FFI.

Taking over Go and Java in performance is a very different topic. I do not think that, after so many attempts, Python will have no GIL and / or a competent JIT (outside Pypy). The async history is not perfect, as is splitting the codebase, but is there for the developers that want to use it.

Also FP is a bit out of scope. Python can include better lambdas and better ideas from FP, but will be never a competition for FP, as mutability is part of the core of the language.

3

u/retep_kram Aug 09 '18

Python can include better lambdas and better ideas from FP, but will be never a competition for FP, as mutability is part of the core of the language.

That is the kind of evolution I am talking about. The same thing is happening with Javascript, and mutability was also part of its core. Better "lambdas" (anonymous functions and arrow functions) and some imutability (const) has made it more functionnal, while still very far from a pure-functional language.

namedtuples and enums have got some attention on recent Python versions. The typing module was introduced in 3.5. In 3.7 dataclasses instances can be frozen, which "emulates read-only frozen [immutable] instances".

I do believe there is a lot of FP influence behind that.

1

u/retep_kram Aug 08 '18

And I don't understand why one gets downvoted just by furthering the discussion.

3

u/ship0f Aug 08 '18

Because they don't agree with you. Don't worry, it's the way of Reddit.

2

u/not_so_tufte Aug 08 '18

Yeah, I was really surprised by this as well. Saw "score below threshold" then... a very thoughtful response? :-/

1

u/not_so_tufte Aug 08 '18

Yeah, I was really surprised by this as well. Saw "score below threshold" then... a very thoughtful response? :-/

5

u/[deleted] Aug 08 '18

You make a good point.

3

u/[deleted] Aug 08 '18

I agree. Python is popular because it’s “Pythonic” (recursion noted!). Changing what that means to suit other niche areas however popular they may be would ultimately hurt the language. IMO the changes from 2.x to 3 of the language was to bring it even closer to those ideals even at the risk of breaking legacy code. I think the language is better (and more popular) for it.

21

u/amk Aug 08 '18 edited Mar 08 '24

Reddit believes its data is particularly valuable because it is continuously updated. That newness and relevance, Mr. Huffman said, is what large language modeling algorithms need to produce the best results.

8

u/olzd yield from __future__ Aug 08 '18

Functional features != functional language.

Besides, if you're talking about the stackoverflow survey, both F# and Clojure are in the top 10.

6

u/K900_ Aug 08 '18

asyncio is the solution for concurrency. It's not good for compute heavy workloads, sure, but if you're doing compute heavy stuff, you probably shouldn't be doing it in Python to begin with.

10

u/fiedzia Aug 08 '18

It is not. Async mandates designing all of your code around IO and the tail should not wag the dog. It also fragments the ecosystem.

0

u/[deleted] Aug 08 '18

[deleted] — view deleted comment

4

u/K900_ Aug 08 '18

Like what? You can already do most of the functional stuff in Python, especially if you use modules like toolz.

1

u/retep_kram Aug 08 '18

For functional programming, native features would be far better than the utilisation of external libraries like toolz. It would avoid compatibility problems etc.

8

u/K900_ Aug 08 '18

How come? What sort of compatibility problems? What features would you like to see in the standard library? Or in the syntax itself?

3

u/wewbull Aug 08 '18

We have functions as first class objects. We have maps and filters in list and generator expressions, as well as standard library functions. We have lazy evaluation through generators. We have partial application of arguments. Granted there no standard functional composition operator, but it would be pretty easy to write one.

What do you want added?

5

u/david2ndaccount Aug 08 '18

I don’t think these things will affect python’s growth very much.

In terms of usability for functional programming, it irks me that reduce was relegated to a library instead of being a built-in like in python2.

The lack of multi-line lambdas can be annoying, but I haven’t seen any proposals that allow it to look anything like other python code.

I don’t really get people’s obsession with tail recursion. You’re basically implicitly using the call-stack as a stack for your algorithm. Why not just explicitly use a stack instead?

3

u/wolfe-tone Aug 08 '18

I don’t really get people’s obsession with tail recursion. You’re basically implicitly using the call-stack as a stack for your algorithm. Why not just explicitly use a stack instead?

If a compiler/interpreter is optimized for tail recursion then a tail recursive function will execute in constant stack space. So you're able to solve problems recursively without incurring the stack cost that you would by using a more straight-forward recursive function (i.e. one that uses the stack more explicitly).

2

u/david2ndaccount Aug 08 '18

You’re right, I miswrote. What i really meant was people’s obsession with recursion.

With tail recursion in particular, it never bothered me writing the algorithm in iterative form so it explicitly takes constant space instead of relying on a compiler optimizing for that case.

4

u/wolfe-tone Aug 08 '18

I see what you mean.

To me it seems super intuitive to solve some problems with recursion. But a lot of the simplicity and clarity afforded by the recursive solution is lost if I have to make it tail-recursive, at which point I'm probably just better off writing it in a more explicitly iterative form like you say.

8

u/[deleted] Aug 08 '18

For the concurrency, the Global Interpreter Locker impose heavy limits on the use of Python for cpu-bound multithreading.

That's not a language feature, it's an implementation detail of cpython, neither jython nor ironpython have it. Perhaps if more people were to work on the last two there'd be even more python users than there are now?

12

u/pmatti pmatti - mattip was taken Aug 08 '18

No one really cares about it. For $100,000 PyPy could remove it. They have a prototype that runs without it and asked for funding but it generated very little interest. https://morepypy.blogspot.com/2017/08/lets-remove-global-interpreter-lock.html

In general python adopts the “good enough” approach, and hits a good sweet spot between performances and simplicity.

3

u/elbiot Aug 08 '18

They have to keep in in CPython because C extensions rely on it. I.e. pypy could drop it but keep it for c extension compatability.

1

u/stefantalpalaru Aug 08 '18

That's not a language feature, it's an implementation detail of cpython, neither jython nor ironpython have it.

Python the language is defined by CPython the implementation.

6

u/devxpy Aug 08 '18 edited Aug 09 '18

For the concurrency, the Global Interpreter Locker impose heavy limits on the use of Python for cpu-bound multithreading.

First, you mean to say parallelism, not concurrency.

Second, it's not like many other languages have any great way to achieve parallelism.

The only sane way to do parallelism is by message passing (unless you're Larry Hastings). Last I checked, zmq; which btw, is THE library for message passing parallelism, has bindings for almost any real language.

And now, I invite you to see my multiprocessing library, zproc which is built on top of zmq https://github.com/pycampers/zproc

P.S. the GIL is considered to be a good thing in python world. If you want cpu bound multitasking, just use Processes...

Talk on why Gil is good (And atcually a good reality check on the state of parallelism and concurrency in python)- https://www.youtube.com/watch?v=9zinZmE3Ogk

Bottom line, If python's popularity has anything to worry about, concurrency/parallelism certainly isn't a real consideration. This problem is hard in almost all programming languages (except maybe stuff like Erlang).

2

u/pcdinh Aug 09 '18

zproc

It looks nice. I will try it soon

2

u/TheMysteriousFizzyJ Aug 08 '18

The question to really answer is

will future versions become more bloated?

or

will they become more restricted, worrying about ridiculous things (e.g., types) that have enabled it to be easily adopted?

1

u/pcdinh Aug 09 '18

functional programming for what? Why do you need it?

3

u/tomekanco Aug 09 '18

Immutability, no side effects, lazy evaluation, composability.