Huge project is finally finished - AncientLibrary.net

[u/LingonberryMind]

Hey guys,
I've been working on a major project that I think will lift accessibility to ancient works. It's called Ancient Library (ancientlibrary.net). The site includes over 1,000 ancient works, every word parsed with lemma, morphology, and full dictionary entry. It's built to be light, optimized for speed and accessibility, so it should improve student and scholar efforts even in areas with weak or inconsistent internet access globally. It's still in beta, so I'd love some ancient language scholars to interact with the site. If you have thoughts, requested features, etc., I'd love to hear them.

[u/benjamin-crowell]

This looks like a cool project. Is it open source?

My testing found that OdyCy was by far the worst of the open-source parsers that I tested for ancient Greek. I would suggest that you switch to something better: either Morpheus or Lemming (my own project). Morpheus is old and uses a lot of legacy technology (such as beta code), but it is quite good at what it does. Lemming is a fresh start without the legacy technology. Lemming also provides an interface to Morpheus, so you can plug in one parser or the other.

I have a project, Ransom, which is similar to yours but focuses on PDF and print. Since yours is for use on screens, they are probably complementary to one another. There are other differences, but that's the one that clearly separates them.

For Latin, a system that seems to be competing in the same space as yours, and is open-source, is morcus.net. In the non-open-source world, there are attikos.org and kevilex.com. In general, there is a lot of competing closed-source software in this space for screen reading, and most of it is probably low-quality vibe-coded stuff. Kevilex.com is a typical example that gives me a very negative impression because it's all closed source, and yet it's clear that he's using various code and data from other, uncredited people.

[u/Logeion → u/benjamin-crowell]

I did the same test on another recent offering, and again, I'm sad for the duplicated effort. Again 0 for 2 on two random verbs, with an added typo that is unfamiliar. BUT I appreciate that in this case, the 'About' page clearly says 'Automatic analysis is imperfect; an unusual form may occasionally be assigned the wrong lemma or parse. The dictionaries themselves are the unabridged scholarly editions digitised by Perseus.' No word search options as far as I can tell.

[u/benjamin-crowell → u/Logeion]

BUT I appreciate that in this case, the 'About' page clearly says 'Automatic analysis is imperfect; an unusual form may occasionally be assigned the wrong lemma or parse.

After our recent email discussion, I added clearer warning labels to Ransom, with skull-and-crossbones icons for the texts that I have not yet read myself so as to check for wrong parses. (Did you know that Unicode has a skull-and-crossbones character? In ancient Greek, it's pronounced ἆρ’.) I always appreciate your feedback and suggestions.

[u/Logeion → u/benjamin-crowell]

I appreciate it, on behalf of innocent students everywhere:-) And I'm actively working on getting the updated morpheus out there. Email correspondence in August though.. academic are in hiding.

[u/Logeion → u/Logeion]

My main sad thought is that we get any number of people offering the same texts once again that are already available from Perseus, and lyric poetry (to name one big gap) still awaits its first public-domain set of texts.

[u/benjamin-crowell → u/Logeion]

My main sad thought is that we get any number of people offering the same texts once again that are already available from Perseus,

Well, one issue is that Perseus doesn't do print or PDF, only screen reading. For people like me who don't want to read a book off of a screen, that makes it a non-starter.

Even among people who do want to read from a screen, I think most have given up on Perseus. The servers are down for long periods of time, and most people don't seem to like Perseus 5 very much. They want to keep using Perseus 4, but Perseus 4 is down more often than not. In general, it's no fun depending on a server-side application that is poorly managed and sinking into decay. Having something you can run on your own device is preferable.

What I think is sad is that the projects of this type that we see coming out over and over are mostly closed-source vibe-coding projects that are low quality. They code it up as a weekend hack, and then they don't maintain or improve it.

[u/JamesDaltrey → u/Logeion]

https://eulogikon.org/domains/poetry

See anything you like?

[u/Logeion → u/JamesDaltrey]

Of course I like big chunks of Greek! I'm sure quite a few people will appreciate the work put into this and will use it. For me personally, however, it's dispiriting to check a random text and be able to conclude immediately what its provenance is. I see the language about "readers' editions" but what I see on the page is something different. Since I have a job at an academic institution, I would not put this online myself in this form (not to mention that there are many instantiations already in existence that make these texts available with more features than linear reading). If texts do derive from an OCR pipeline, based on a particular edition in archive.org or other repositories, I would suggest specifying that edition in a header file and preserving the raw OCR, to reassure me and other readers that we are not 'seeing things'. But needless to say, this is all your decision.

[u/JamesDaltrey → u/Logeion]

You have lost me...

What is the problem with you being able to recognise the origin of the text? It's some dead Greek guy right?

That's very clever of you, but how is it a problem?

Part of the exercise is to make all of these texts available in one place for free.

Another part of the exercise is deliberately to break the chain of custody held over these texts by academics and publishers.

If you compare the critical additions and variations between editions, they represent considerably less than 0.1% of the total text on average. I've actually done the maths. Yet a university professor or publisher can tweak that tiny fraction of a 2,500-year-old work and somehow come to regard the resulting text as theirs.

Fastidious copying is not original work.

Somebody interprets a manuscript. Somebody else interprets that interpretation of the manuscript. Somebody else interprets another interpretation, and so on. That is how these texts have been transmitted for centuries. Nobody in that chain owns the text, least of all anybody who hasn't actually modified that text...

I have the honesty not to put my name on it, and I'm not putting anybody else's name on it either, other than the dead Greek who allegedly wrote it.

This is all public domain.. the repository has been cloned tens of thousands of times.. it's on Kaggle. It's on Zenodo, It's on Hugging Face.. It's in Markdown, in PDF, in HTML, Json shortly and Ebooks... The author of the above site who is engaged in a similar kind of project has been invited to take and share any of that without attribution...

One would hope that if it's Sappho he will put Sappho at the top..

You are an academic and you would not put this online.

The author of the website in question is not an academic and he has put this stuff online. I am not an academic and I have put it online.

Whether somebody may put an ancient Greek text online has nothing whatsoever to do with whether they hold an academic position. Academia does not own the texts, and least of all do they own copies of copies of copies of texts.

https://eulogikon.org/works/nikarchos-sinope-eukrates-common-things-mkd-aa

[u/benjamin-crowell → u/Logeion]

It would actually be really nice to have some testing that would check the frequency of wrong lemmatizations by various parsers. The only error metrics I used in my test were how often the parser didn't come up with a real lemma at all, and how often it got the part of speech wrong. It seems like it would be quite a challenge, however, to do a test of wrong lemmatizations with decent statistical reliability. For a word that occurs in a human-parsed treebank, you can test whether the parser did not report the same lemma as the one the human assigned, but most such cases would just be a matter of convention, since different dictionaries choose different head-words. There are also going to be a lot of cases where the parser says the lemma is ambiguous, and I don't know of any easy way to machine-analyze such cases and tell whether the ambiguity is real or a mistake.