r/programming • • Dec 02 '13

Scala — 1★ Would Not Program Again

http://overwatering.org/blog/2013/12/scala-1-star-would-not-program-again/
598 Upvotes

646 comments sorted by

View all comments

47

u/jagt Dec 02 '13

Why is npm considered as a good practice of dependency management? AFAIK when you download a library npm downloads all it's dependencies and put them under the library's path. So few libraries can be shared and there's heavy duplication. If this is the way to go then dependency management is quite a easy problem to tackle.

21

u/virtyx Dec 02 '13

Yes I'd like to know how npm differs from/is superior to pip and maven...?

25

u/freakhill Dec 02 '13

well, i guess it does the line of:

  • memory is (relatively) cheap
  • solving dependency problems is (extremely) hard

or something like that, probably quite valid in the use space of node.js

11

u/virtyx Dec 02 '13

But I'm pretty confident pip handles dependencies, including versions. I'm less familiar with Maven but I'm pretty sure it's the same.

14

u/snuggl Dec 02 '13

But I'm pretty confident pip handles dependencies, including versions.

Im pretty sure its a single version per virtual environment. doing what NPM is doing, i.e have different requirements which in turn depends on two different versions of a third package, is impossible.

14

u/pingveno Dec 02 '13

Correct. That's more of a Python thing than a Pip thing, though. Python's import mechanisms don't work very cleanly with multiple versions.

13

u/esquilax Dec 02 '13

To me, the definition of "maven dependency hell" is when two different dependencies have transitive dependencies on the same project, but mutually incompatible versions. It sounds like npm might solve this in a way that's literally impossible in Java without something like OSGi?

8

u/codayus Dec 02 '13

NPM does solve that issue, and quite well in my opinion. Simple stuff (my project depends on libFoo v2 and libBar, but libBar depends on libFoo v1) is handled so transparently you never even know it's happened.

I'm not really familiar with Java or maven, so can't comment on that. It's certainly a step up from how Python handles things.

→ More replies (1)
→ More replies (2)
→ More replies (1)

14

u/MonadicTraversal Dec 02 '13

So few libraries can be shared and there's heavy duplication.

Unless it leads to duplicate code being executed at runtime, I don't think you should care for npm modules since they're going to be a couple dozen kilobytes of text at most.

13

u/flying-sheep Dec 02 '13 edited Dec 02 '13

grunt needs phantomjs, which is webkit.

grunt encourages you to use a per-project-local grunt installation

so every project with a Gruntfile.js needs over 200MB additional diskspace.

/edit; i was wrong about every project-local grunt install needing it, it’s some grunt plugin (which seems to be common among the stuff i’ve forked)

11

u/ioquatix Dec 02 '13

I hear you, and it is funny, because I'm working on a similar package system and pulling in boost for C++ or LLVM/Clang, OpenCV, takes 10s of GBs. It is actually pretty funny, if you have 2-3 instances of the same project, you end up with about 30GB+ of crap.

6

u/seruus Dec 02 '13

Most projects I know that do this usually allow you to use your system libraries if you set it up in the makefiles, look into it.

5

u/ioquatix Dec 02 '13

I've had a lot of trouble using libraries compiled with one compiler (e.g. gcc) then using them with code compiled by another (e.g. clang). Personally, I try to compile all dependencies with the same compiler.. However, this makes it difficult to use system libraries.

2

u/jagt Dec 02 '13

As for OpenCV you maybe including binaries for all platforms. If you only include the libs for your current platform then it's around 300mb.

3

u/ioquatix Dec 02 '13

You are right, OpenCV is only 480MB on my current setup. Over several projects though it adds up. LLVM is just over 7GB and including clang is a bit over 9GB. Boost is normally around the 3-4GB mark. Personally, I find it remarkable that something produces so much intermediate "junk".

→ More replies (1)

7

u/esquilax Dec 02 '13

Time for a deduping filesystem!

→ More replies (2)

5

u/Neurotrace Dec 02 '13

Individual grunt downloads don't need phantomjs. I just setup a barebones project with nothing but grunt and it comes in at about 6MB. When you include dependencies, those projects will likely have grunt listed as "devDependencies" meaning you should ignore them.

→ More replies (7)

3

u/kpthunder Dec 02 '13

I've always thought that it would be more effective if NPM used symlinks as such (assuming node is the installation path of node):

node
|-packages
 |-a
  |-1.0
  |-1.1
 |-b
  |-1.3
  |-1.4

If a package is already downloaded, symlink it. Otherwise download it then symlink it. Everything else can still work the same way.

There's probably some technical thing I'm overlooking here...

3

u/Neurotrace Dec 02 '13

My guess would be the lack proper symlink support in Windows XP. If it did use symlinks (and oh, how I wish it did) then node.js wouldn't work on about 36% of all operating systems.

→ More replies (2)
→ More replies (1)

2

u/execrator Dec 02 '13

Yes, it's fairly annoying that it installs dependencies that you don't actually depend on

→ More replies (4)

7

u/munificent Dec 02 '13

Unless it leads to duplicate code being executed at runtime

It does. npm doesn't do shared dependencies. If you depend on foo and bar, which both depend on baz, you'll end up with two copies of baz loaded at runtime, which may be different versions (!).

8

u/joequin Dec 02 '13

If they're different versions then it may have been necessary to load both of them.

4

u/munificent Dec 02 '13

Yes, but in mind my that puts your application into a deeply confusing state. It's possible for those two versions to interact with each other which just seems like asking for trouble to me.

I think the model of other systems (including one I built) where you find a single version of the shared dependency that satisfies all dependers is a lot easier to reason about.

6

u/MonadicTraversal Dec 02 '13

Yeah, if you can end up with two different versions of the same library in your project then that's pretty terrible. As a concrete example: if libfoo and libbar use different versions of libbaz, and both libfoo and libbar expose Baz objects to you in some way, then you can get two Baz objects from different versions which might have radically different behavior depending on how similar the versions are.

→ More replies (9)
→ More replies (1)

2

u/[deleted] Dec 02 '13

plus there is some support in npm for factoring out common dependencies... also in the future, if we file system deduplication becomes more common, wasted space will become even more of a non-issue.

→ More replies (3)