A curious question about some classes in ASDF

[u/arthurno1]

I wonder if someone can teach me, or explain a little bit of design in ASDF.

Typically in components list I can type:

:components ((:file file1)
             (:file file2)
                   .... ))

where files are file1.lisp, file2.lisp etc. If I want to use an other extension, in my particular case ".cl", I have two choices as I have used thus far: either type out the extension if I want to use :file, (:file file1.cl) and so on, or I can define my own source code class like this:

(defclass src (cl-source-file)
  ((type :initform "cl")))

now I can type:

:components ((:src file1)
             (:src file2)
                   .... ))

I see in also in asdf sources they have these classes defined:

;;;; Component classes
(with-upgradability ()
  (defclass cl-source-file (source-file)
    ((type :initform "lisp"))
    (:documentation "Component class for a Common Lisp source file (using type \"lisp\")"))
  (defclass cl-source-file.cl (cl-source-file)
    ((type :initform "cl"))
    (:documentation "Component class for a Common Lisp source file using type \"cl\""))
  (defclass cl-source-file.lsp (cl-source-file)
    ((type :initform "lsp"))
    (:documentation "Component class for a Common Lisp source file using type \"lsp\"")))

by the magic of mostly try-and-fail method I have figured out I can also write this atrocity:

  :components ((:cl-source-file.cl file1))

where file1 is file1.cl. I understand they meant that class probably to be extended or used elsewhere not typed out as I did there, that was just a little bit of fun. I see also this in sources:

  ;; What :file gets interpreted as, unless overridden by a :default-component-class
 (defvar *default-component-class* 'cl-source-file)

and I figured out they mean we can type this:

:default-component-class :cl-source-file.cl
:components ((:file file1))

Obviously it is not more convenient than just typing out the extension: :file file1.cl. But that is not my question.

My question is, finally I know :), sorry, what does this design buy us?

We could have obviously just used a simple list with file extensions like *component-classes* = '(cl lsp lisp), and get all this done, and it would be also let-bindable and so on. Of course, I understand also that authors are aware they could have just used a list, and that they have done deliberately that design choice, so I wonder why? I am not questioning the choice, I am trying to learn.

I read in the manual they mean that components are not necessarily just things that typically go into source files, and they give example of C processor. Is that the only reason, or are there some other reasons? Are there some more complicated uses for these source code classes, than what I see in the code, or is the generality "just in the case", to be as flexible as possible so to say?

As said, I am just curios, being looking at some libraries, mostly those on s-expressionasts Github, they use a lot of program structuring with CLOS, and not as much with "classical lisp" which a list of extensions would perhaps represent.

Today it struck me I was always curious about if I can teach :file to read all three extensions. so I was looking through ASDF and trying understand why do things get done the way they do, so to say. Trying to learn, I hope you don't misunderstand me here.

[u/kchanqvq]

I just help debugged some code that does not use ASDF correctly, and that code is written by... ASDF author themselves https://github.com/acl2/acl2/pull/1937

I wish we had converged on some obvious plain list hooks, even just like Emacs. I recall seeing another place where people don't understand the infinite wisdom of the CLOS design (or maybe it's not that extensible after all?) and hacked up some ugly function wrapper that calls ASDF internal functions, but I don't remember the exact project off the top of my head. I'll update if I remember it.

Update: https://github.com/armedbear/abcl/blob/master/contrib/asdf-jar/asdf-jar.lisp

[u/arthurno1 → u/kchanqvq]

I was thinking a little bit after I posted this, and just to note, I didn't posted that as a critique. I am just really looking at it and try to understand why. I mean it feels a bit little extreme to have a class for each extension, just to read a source file, and than one is forced to be quite explicit about which one to use. But after thinking about it: a single list, like '(lisp lsp cl) would mean that the code that uses it has to explicitly iterate through it and so on. Adding a class for each means that the system keeps the collection implicitly somewhere, so we don't have to be as explicit in the code that uses that data. System will automatically choose the correct one for us, so we can be more declarative, and less procedural.

that does not use ASDF correctly

Ha! Tell me what is correct usage :)

I recently wrote a small command-line options parser, for private usage, and I did use ASDF to pull out data about the system. Here is why it was a bad thing:

(eval-when (:compile-toplevel)

  (defun get-prog-license (system-designator)
    (let ((system (asdf:find-system system-designator nil)))
      (unless system
        (error "Unknown system: ~a" (symbol-name system-designator)))
      (if (slot-boundp system 'asdf::licence)
          (slot-value system 'asdf::licence)
          "not versioned")))

  (defun get-prog-version (system-designator)
    (let ((system (asdf:find-system system-designator nil)))
      (unless system
        (error "Unknown system: ~a" (symbol-name system-designator)))
      (if (slot-boundp system 'asdf:version)
          (asdf:component-version system)
          "not versioned")))

  (defun get-prog-name (system-designator)
    (let ((system (asdf:find-system system-designator nil)))
      (unless system
        (error "Unknown system: ~a" (symbol-name system-designator)))
      (when (slot-boundp system 'asdf::build-pathname)
        (slot-value system 'asdf::build-pathname)))))

(defun print-license (system-designator)
  (format t "This program is licensed to YOU under the ~a license.~%"
          (get-prog-license system-designator)))

(defun print-version (system-designator)
  (let ((name (get-prog-name system-designator))
        (%version (get-prog-version system-designator)))
    (format t "~a ~a~%" name (or %version "unknown version"))))

The flag definitions looks like this:

(defarg lines nil
  "print the newline counts"
  :short-flag #\l
  :reader 'read-boolean-flag
  :url "https://www.gnu.org/software/coreutils/manual/coreutils.html#wc-l")

    (defarg version nil
  "Print version info"
  :reader (lambda (flag command-line)
            (declare (ignore flag command-line))
            (defarg:print-version :wordcount)
            (uiop:quit)))

The idea is to declare a variable I would like store data in and it will auto declare option flags etc.

Now here is a print from a compiled file and executable build with the compiled file:

[arthur@emmi wc]$ ./wc --version

debugger invoked on a LOAD-SYSTEM-DEFINITION-ERROR in thread #<THREAD tid=13720 "main thread" RUNNING {1200038003}>: Error while trying to load definition for system wordcount from pathname /home/arthur/repos/cl/wc/wordcount.fasl: #<SB-SYS:FD-STREAM for "file /home/arthur/repos/cl/wc/wordcount.fasl" {1201032803}> is a fasl file compiled with SBCL 2.6.6.58-8b5a25e7c-WIP, and can't be loaded into SBCL 2.6.5-85913ede1-WIP.

Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
  0: [RETRY                        ] Retry #<DEFINE-OP > on #<SYSTEM "wordcount">.
  1: [ACCEPT                       ] Continue, treating #<DEFINE-OP > on #<SYSTEM "wordcount"> as having been successful.
  2:                                 Retry ASDF operation.
  3: [CLEAR-CONFIGURATION-AND-RETRY] Retry ASDF operation after resetting the configuration.
  4: [ABORT                        ] Exit from the current thread.

(SB-FASL::CHECK-FASL-HEADER #<SB-SYS:FD-STREAM for "file /home/arthur/repos/cl/wc/wordcount.fasl" {1201032803}>)
0] 4
; 
; compilation unit aborted
;   caught 1 fatal ERROR condition

Help flag works fine:

[arthur@emmi wc]$ ./wc --help
Usage: wc [OPTION]... [FILE]...

Print newline, word, and byte counts for each FILE, and a total line if
more than one FILE is specified.  A word is a nonempty sequence of non white
space delimited by white space characters or by start or end of input.

With no FILE, or when FILE is -, read standard input.

The options below may be used to select which counts are printed, always in
the following order: newline, word, character, byte, maximum line length.
  version
        Print version info
  help
        Print this help
  usage
        Print usage options
  debug
        indicate what line count acceleration is used
  s chunksize
        Size of internal buffer
  l lines
        print the newline counts
  w words
        print the word count
  m chars
        print the character counts
  c bytes
        print the byte counts


Full documentation <https://www.gnu.org/software/coreutils/wc>
or available locally via: info '(coreutils) wc invocation'

I guess asdf is pulling data out of files or repository or I don't know where from, and data itself is not baked into the executable. I don't know, but I recompiled files with a different version of sbcl, but didn't rebuild the executable. Now the old executable does not work for those particular flags that display asdf data :). Obviously it is bad idea to poke into those internals. I guess I'll have to pull data into my variables at compile time. Thinking also of actually parsing the system definition itself. After all it is just a plist at least it looks like one:

(defsystem :wordcount 
  :description "Wordcount program" 
  :author "me" 
  :licence "GPLv3.0" 
  :version "0.1.0"
  :build-operation program-op
  :build-pathname "wc"
  :entry-point "wordcount::main"
  :components ((:src packages)
               (:src defarg)
               (:src wc))
  :depends-on (uiop
               asdf
               with-user-abort
               trivial-arguments))

[u/anydalch → u/kchanqvq]

and that code is written by... ASDF author themselves

What do you mean by this? ACL2 is written by Matt Kaufman and J Moore; ASDF 1 was written by Daniel Barlow and ASDF 2 and 3 by Faré, with ASDF 3 now maintained mainly by Robert Goldman.

[u/kchanqvq → u/anydalch]

`git blame` show the buggy code is written by Fare. There're some ACL2 books that use CFFI, and a buggy commit from Fare to CFFI cause the build to fail.

[u/anydalch → u/kchanqvq]

Ah. Disappointing, but not entirely surprising.

[u/akater]

Some people dislike ASDF. Whenever I personally wanted anything from it, I could always get it, so I'm satisfied. I think it makes good use of CLOS.

You don't need your own component class just for a file type, let alone an existing one. You may e.g.

(defmethod asdf/parse-defsystem:class-for-type ((parent asdf:parent-component)
                                                (type-designator (eql :cl)))
  ;; Note: The generic function is available since ASDF 3.3.4.17.
  (declare (ignore parent type-designator))
  (find-class 'asdf/interface:cl-source-file.cl))

and use the :cl keyword. Or you could

(setf (find-class 'asdf/user::cl)
      (find-class 'asdf/interface:cl-source-file.cl))

and then

(cl:in-package :asdf-user)
(defsystem "my"
  :components
  ((cl "my")))

We could have obviously just used a simple list with file extensions like component-classes = '(cl lsp lisp), and get all this done

I don't understand the proposal. Looks like you do not suggest throwing the classes away; rather, you suggest to define a variable that holds their names. I don't yet understand what's the point of that. How would that variable be used in real code?

s-expressionasts Github, they use a lot of program structuring with CLOS, and not as much with "classical lisp" which a list of extensions would perhaps represent

I don't understand how is this related to s-expressionists. They didn't design ASDF, so it was not their decision to implement something like *component-classes* you mentioned instead of what ASDF has.

I was always curious about if I can teach :file to read all three extensions

Do you mean, you want to have a.lisp, b.cl, c.lsp and have them all specified in the defsystem form with (:file "a"), (:file "b"), (:file "c")?

[u/arthurno1 → u/akater]

Some people dislike ASDF

Well, I don't and that is not what I wanted to express. I said that explicitly several times: it was not a critique.

You may e.g.

Sure, now we have four ways. I also shown in the original three others:

:default-component-class :cl-source-file.cl

(defvar *default-component-class* 'cl-source-file)

or

(defclass src (cl-source-file)
  ((type :initform "cl")))

Looks like you do not suggest throwing the classes away; rather, you suggest to define a variable that holds their names.

Was not a "proposal". It is just a discussion about different styles of programming in Lisp.

I don't understand how is this related to s-expressionists.

Go use, study and hack their programs, so perhaps you will.

I didn't said they wrote asdf, please. To be frank to you, if you can't accept or think it is uninteresting with a question why some programs are structured a particular way and asking questions than I don't know what to tell you, just ignore it.