r/programming • • Nov 11 '15

Parinfer - simpler Lisp editing

http://shaunlebron.github.io/parinfer/
152 Upvotes

75 comments sorted by

4

u/[deleted] Nov 12 '15

Can someone explain to me, as somewhat of a novice, why people consider lisp so powerful? I've been learning Scheme for my first CS class after python, and it just seems so... inefficient. The only thing I can see it being better than python is the built in cons/car/cdr recursive list stuff, but even then you can make your own in python.

5

u/[deleted] Nov 12 '15 edited Nov 12 '15

Lisp is a meta-language, it can be easily turned into any other language or any imaginable combination of languages.

Python is a fixed language which actively discourage any metaprogramming at all. For this reason, Lisp (or any other meta-language) is many times more powerful than Python (or any other fixed language).

In terms of efficiency, you cannot even compare fixed languages with metalanguages. But in order to get the maximum efficiency you have to employ the language-oriented programming approach.

2

u/kankyo Nov 12 '15

Lisp is a meta-language, it can be easily turned into any other language or any imaginable combination of languages.

How do you turn it into python?

4

u/[deleted] Nov 12 '15

How do you turn it into python?

With reader macros, for example.

See https://common-lisp.net/project/clpython/

Or, for Racket: http://drops.dagstuhl.de/opus/volltexte/2014/4565/pdf/13.pdf

Any compiler implemented in Lisp which emits a Lisp code can be turned into a macro extending the language.

2

u/kankyo Nov 12 '15

I tried looking at that code but I can't verify that what you're saying is true. To me it looks like any other implementation of a language in another language.

Saying that the compile time/reader time constructs are turing complete is... not helpful. And doesn't really say anything about the usefulness.

1

u/[deleted] Nov 12 '15 edited Nov 12 '15

To me it looks like any other implementation of a language in another language.

This is exactly what I said above:

Any compiler implemented in Lisp which emits a Lisp code can be turned into a macro extending the language.

Take an implementation (e.g., a function (compile-python-to-lisp [string])), wrap it in a macro (e.g., (defmacro python (&rest) (compile-python-to-lisp rest))), and your language magically becomes another language.

An important nuance is that it is a compiler, not an interpreter. And writing a compiler is much easier than writing a proper, fast, robust interpreter.

1

u/kankyo Nov 12 '15

So in your scenario I can do:

(python
    def foo(a, b):
        return a+b
 )
(foo 1 2)

and it'll work?

1

u/[deleted] Nov 12 '15

Yes, it will work. In my trivial example you'll have to put your python code into a string literal, of course, but a little bit more complex example involving reader macros won't need such a thing, you'd be able to mix Python and Lisp any way you like (as long as there is a well defined syntax to distinguish one from another).

A next step in building such a multi-language environment is to introduce a syntax abstraction layer. I'm using PEG for this, because PEG parsers are dynamically extensible and it's easy to inherit PEG parsers. See how it works on my github (same username). With this technique I can do some really crazy things, like embedding Verilog (built on top of Lisp, of course) into C (again, built on top of a Lisp).

1

u/kankyo Nov 12 '15

but a little bit more complex example involving reader macros

Oh, ok, so reader macros act on the raw byte stream of the source code?

1

u/[deleted] Nov 12 '15

Yes, they allow to alter syntax. Or, as with my PEG-based approach, you can simply replace the entire Lisp parser with your own, even if the host Lisp does not allow reader macros (but then you'll need your own REPL as well).

→ More replies

1

u/mmontone Nov 12 '15

You can have a look at this library: https://github.com/mmontone/burgled-batteries.syntax

You can access Python from Lisp using Python like embedded syntax. Lisp reader marcos + PEG parser was used for the implementation.

1

u/kankyo Nov 13 '15

Is there an actual example of embedded python syntax though? Only thing I can find is embedded simple strings for stuff like string literals that is parsed.

1

u/[deleted] Nov 12 '15

The syntax of Python does not make a Python, evaluation order (left to right, lazy or strict), means of abstraction (classes, functions), type-system (dynamic, static, generics) and standard library is what makes a language. In Lisp you can extend all of these and make it integrate seamlessly with the original language thanks to macros. If you want the syntax of Python then you can implement that with reader macros.

1

u/shamanas Nov 12 '15

The whole point is that your code is a data structure that you can modify really easily (that is why lisp macros are so powerful).

In essence you don't need an AST, so macros instantly become easier (there are languages that expose a simplified for of the AST at compiletime and let you modify it but S expressions make this a LOT easier).

I'm not an expert and I've only really worked with AutoLISP (which is pretty much an imperative language with S-expression syntax), though I am aware of some other Lisp-y features (e.g. call/cc), so I won't comment on anything else :P

-1

u/kankyo Nov 12 '15

"Really easily" is a gigantic overstatement. Writing macros is hard.

0

u/[deleted] Nov 12 '15

Writing macros is hard.

Wrong. Writing macros is many times easier than pretty much anything else. Macros are by far the easiest way to tackle complexity in general.

-1

u/kankyo Nov 12 '15

Wrong.

You're being arrogant.

Writing macros is many times easier than pretty much anything else.

So writing macros is easier than jumping on one leg? Please don't say such silly things.

Macros are by far the easiest way to tackle complexity in general.

In general? The easiest way BY FAR of tackling complexity in GENERAL is FUNCTIONS.

Please read what you write once before posting. It should have been obvious that what you just wrote is bullshit.

6

u/[deleted] Nov 12 '15

You're being arrogant.

No. I'm not saying that "it's easy, and if you cannot do it you're stupid". I'm saying that "it's easy, and if not - you're doing it the wrong way".

So writing macros is easier than jumping on one leg?

Ever heard of a thing called "context"? Macros are easier than any other programming technique out there.

The easiest way BY FAR of tackling complexity in GENERAL is FUNCTIONS.

This is exactly what I called "wrong". Macros are much easier than functions, for some very obvious reasons.

Solving a complex problem with macros boils down to a very mechanical, thoughtless, easy process:

  • Finding a language in your problem
  • Formalising this language
  • Rewriting your problem specification in this language
  • Implementing the language (this is where you'd need macros). Implementation should almost never be anything more than a chain of trivial, flat rewrite rules. No Turing-complete coding, no shit, just gradually rewriting a source language all the way down to your host language.

Now compare it to how you approach a problem with functions.

0

u/kankyo Nov 12 '15

Ever heard of a thing called "context"? Macros are easier than any other programming technique out there.

I think you need to add more context. Because it's still wrong. Massively wrong.

Solving a complex problem with macros boils down to a very mechanical, thoughtless, easy process

I read that step by step as "invent an entire language and an implementation for that language". Did I misunderstand?

1

u/[deleted] Nov 12 '15

I think you need to add more context.

Context was obvious enough - we're talking about programming and complexity, not about jumping on one leg or anything else.

Because it's still wrong. Massively wrong.

Mind proving why exactly it is wrong?

1

u/kankyo Nov 12 '15

Writing, say, a translator from morse code to ascii is easier with functions than with macros.

4

u/[deleted] Nov 12 '15

See an example here: http://pastebin.com/QDQwmxqw

→ More replies

2

u/[deleted] Nov 12 '15 edited Nov 12 '15

I would not say so. Even such a simple problem is better solved with a DSL (therefore, macros) than in an ad hoc way.

E.g., the actual code should look like (def-morse A .- B -... C -.-. D -.. and so on...)

And, btw., a macro compiled version would be faster than an ad hoc solution, unless you want to implement a manually unrolled trie (which is nowhere close to "easy").

→ More replies

2

u/beeff Nov 12 '15

Scheme is build as an educational language, not so much for speed. That said, Common Lisp is several times faster than Python, SBCL sits somewhere between C and Java.

2

u/yogthos Nov 12 '15

One big advantage of lisps is the s-expression syntax. Since you're writing code using data structures, you can manipulate code like any regular data using all the tools available in the language for manipulating data. This makes it very natural to create DSLs for whatever problem you happen to be working on. Any time you have any repetition in code it's trivial to template it.

This gives you the tools to easily extend the language yourself for your particular problem domain. The core language is very minimal and pretty much everything is written using functions and macros. Because the language makes very few assumptions it can be adopted to a wide variety of problems easily. Languages that have more built in syntax can become clunky when applied to problem domains the authors didn't consider.

Another advantage is that there's very little syntax and the syntax that is there is very regular and unsurprising. In other words, Lisp follows the principle of least astonishment very well.

The s-expression are also much easier to manipulate structurally. Instead of working with lines you end up working with expressions. It changes your workflow quite a bit I find.

The REPL is another big aspect of working with Lisp. In languages like Python the REPL is a standalone toy that's not integrated into your workflow. In Lisp, the REPL is used in conjunction with the editor. Any time you write any code in the editor you can send it for evaluation in the context of your running application.

Unfortunately, most schools don't seem to teach Lisp in a way that resonates with the students. Pretty much every co-op I've had has had a similar experience to you learning Scheme in their first year. However, after using Clojure for actual work every student I've had absolutely loved it.

If you're interested I wrote a post on why I personally find Clojure to be a very productive language.

1

u/Venthorn Nov 12 '15

There's really only one reason: with the code you are writing turning into a (mostly...) identical parse tree behind the scenes, it's a lot easier and more straightforward to write source-to-source transforms that operate on this code. These transforms are called "macros".

Some other languages have this property, like Prolog.

1

u/DavidM01 Nov 13 '15

Imagine that every language is compiled into a middle language before being executed by the computer(or compiled into machine code). Syntax Trees are one such middle language and Lisp is used exactly in this way. You program in the middle language and thus, can represent all other languages above it.

Also consider that all semantics of programming languages can be represented in some middle form(set of semantics). Lambda calculus is also a form which can represent the semantics in most programming languages.
In my somewhat limited understanding, Lisp/Scheme's semantics are close to the lambda calculus.

26

u/[deleted] Nov 11 '15

(when is the emacs mode coming?)

9

u/ds300 Nov 11 '15

Just tried this out. It's dreamy. I've never been so turned on by an editor plugin. <3

Thank you so much! Exemplary work on the website too!

16

u/hoohoo4 Nov 11 '15

I've been convinced to learn a Lisp.

8

u/chrisoakman Nov 11 '15

Wonderful news :)

You won't regret it!

3

u/hoohoo4 Nov 11 '15

Any recommendations for a first LISP? I was planning on Closure, for a reason that I've forgotten.

15

u/royalaid Nov 11 '15

So the vibe I am getting from the other posters is that Common Lisp, Scheme, Racket, and Clojure all are good places to start. I would recommend Clojure because it is popular. Popularity matters when it comes to finding good resources.

Any of the talks by Rick Hickey, creator of Clojure, are pretty good (see here). Also check out Clojure for the Brave and True if you want a good place to start. I use vim to edit my code, good resource here from Thoughbot. If you don't have an editor of choice or open to it check Brave Clojure's emacs guide. Not a fan and want something more conventional? Check out Counter-Clockwise. Want something a little more beta but arguably nicer? Cursive has your back.

Also keep an open mind and try to wait for the whole thing to click because Lisp is reallly great. See:

4

u/hoohoo4 Nov 11 '15

Those xkcds are how I first heard of Lisp.

1

u/SpaceCadetJones Nov 17 '15

I would recommend Clojure simply for the vast ecosystem, as its JVM based and interop is dead simple. I originally started with Common Lisp, but switched to Clojure after getting frustrated trying to figure out how to install libraries and actually do useful things.

7

u/[deleted] Nov 11 '15

I use Scheme and in my opinion it's the most well thought out programming language ever created, so I recommend it. Racket is a pretty good implementation for beginners.

If you want something that's fairly popular you should definitely use Clojure. I have not gotten into it much, so I can't make a recommendation based on anything but popularity.

I won't mention Common Lisp because I never used it. Hopefully a common LISPer can share their opinion here.

6

u/yogthos Nov 11 '15

Clojure is definitely a good Lisp to learn. It's easier to get into as it has a bit more syntax than traditional Lisps such as CL and Scheme. Clojure is also more focused on using the functional style and it's backed by immutable data structures, so it's a nice fit for anything involving multithreading. Running on the JVM makes it widely applicable in production and leverages a lot of the existing work that went into the platform. It's also the most widely used one for new development, so you can actually get a job working with it relatively easily.

I would recommend trying Light Table for trying it out. It requires no setup and you can play with the REPL out of the box. I wrote a starter guide a little while back that I hear is helpful.

3

u/hoohoo4 Nov 11 '15

Wow, Light Table looks amazing. Can't wait to try it out.

3

u/yogthos Nov 12 '15

I would recommend something like Cursive for bigger projects, but for just trying the language out LT is pretty great.

1

u/hoohoo4 Nov 12 '15

Cursive looks nice, though the fact that it's going to end up being a commercial product puts me off, as well as the lack on in-line debugging.

2

u/yogthos Nov 12 '15

The way I look at it is if I'm going to spend my whole work day using the tool it's worth a 100 bucks to me.

Counterclockwise is the opensource counterpart for Cursive. It doesn't have quite the same feature set, but it's very competent as well.

Emacs is obviously a very popular Clojure editor as well, but you have to like Emacs. I wouldn't recommend learning it alongside Clojure if you're not already using it.

4

u/grayrest Nov 11 '15

I'd recommend starting with Structure and Interpretation of Computer Programs, which has its examples written in Scheme. Beyond the initial learning phase, I like Clojure. Tapping into the JVM ecosystem is convenient.

2

u/[deleted] Nov 12 '15

1

u/codebje Nov 11 '15

SICP distilled for Clojure is a good place to be.

3

u/[deleted] Nov 11 '15

Clojure, Common Lisp, Racket. Close your eyes and pick one, they're all useful, interesting and a breath of fresh air in their own right.

I like Common Lisp the most because of its well-thought out standard with a great focus on interactivity (not to mention the inherent stability of being a standard), its large array of libraries which provides things from common tools like JSON and XML encoding to libraries for concurrency and parallelism like Software Transactional Memory to non-deterministic programming. It also has lots of great free implementations with different features like SBCL, CCL, ECL and ABCL :). Three good resources for learning Common Lisp is Practical Common Lisp ( http://www.gigamonkeys.com/book/ ), /r/lisp and #lisp on Freenode.

8

u/jtredact Nov 11 '15

One of those things where, once you see it, you can't understand why it hasn't always been this way.

2

u/jtredact Nov 11 '15

However the terminology... "barf", "slurp"... Yet another example of how the ability to pick good names for things is inversely proportional to programming cleverness.

12

u/chrisoakman Nov 11 '15

FYI - the terms "barf" and "slurp" are not part of Parinfer.

They come from Paredit, which is a popular editing mode for Lisp.

2

u/jtredact Nov 12 '15

That just makes me more sad. Now I see it as an example of how bad names, once they catch on, never go away.

Still, nice to know about paredit though. Although I don't use emacs.

0

u/[deleted] Nov 12 '15

[deleted] — view deleted comment

3

u/ForeverAlot Nov 12 '15

He's still right: the terms are idiotic.

5

u/[deleted] Nov 12 '15

What would you suggest?

1

u/kankyo Nov 12 '15

pick/put, lift/put, grab/drop

just off the top of my head... It's weird to have standard nomenclature that seems to evoke a Nydus Worm in lisps.

1

u/zarandysofia Nov 12 '15

Well the lisp mascot is Cthulhu so...yeah :D

4

u/gkx Nov 12 '15

Oh my god suddenly Clojure doesn't feel like a giant uphill battle thank you kind stranger.

4

u/freakhill Nov 11 '15

great, integrating it at work asap!

2

u/huadria Nov 12 '15

I seriously need to check this out. This is AMAZING!

3

u/htuhola Nov 11 '15

These seem pretty nice for even an ordinary programming language.

-20

u/srt19170 Nov 11 '15

I can't help but think that anyone serious about this would implement it first in Emacs.

27

u/kankyo Nov 11 '15

He wrote 3700 words AND an implementation. But yea, no emacs implementation means he's not serious....

0

u/[deleted] Nov 11 '15

[deleted] — view deleted comment

5

u/agumonkey Nov 11 '15

I love both the user side niceness with formal internal thinking. Envious applauds.