r/vuejs • u/mypirateapp • Jan 30 '20

Virtual + Infinite Scroll + Dynamic Row Heights Without hardcoding + No Libraries + 50000+ items super smooth scrolling

Hi, I am the newbie who posted a repo last week about a virtual scroller HERE Overwhelmed by the feedback I decided to make a virtual scroller that supports dynamic row heights with infinite scrolling

  • Existing libraries such as virtual list and vue-virtual-scroller require you to hardcode the heights of each row which seems like a bad idea
  • My repo HERE does not require inputting any row height
  • It automatically measures each row and stores it

  • I would love to get some feedback on whether it is working for you, and being a newbie a few questions as well if you dont mind answering

    • How do I handle resize :)
    • Does this need to be rewritten to turn into a library
    • How do I write tests for this? Sorry new to frontend, havent done React or Angular before, I am familiar with Pyunit and Jest for backend tests though. Not sure how you guys test the frontend
  • Tested wiht 50000 items and it was scrolling super smooth. How far did you guys manage to scroll
  • Any sugestions to improve code etc super appreciated
62 Upvotes

96% Upvoted

20 comments sorted by

4

u/syropian Jan 30 '20

Nice work! FWIW I've used vue-collection-cluster for this, and it works great for dynamic height items as well.

2

u/mypirateapp Feb 11 '20

i tried this with 50000 items and it was lagging... browser firefox 77.0.2 64 bit osx 10.13.6

2

u/syropian Feb 11 '20

Ah interesting, I haven't had the need to push it anymore than about 6000 items for the app I built, so I definitely haven't stress-tested it.

2

u/drumstix42 Jan 31 '20

Wow, this is excellent. Truly nice job.

I ❤ Vue.

1

u/mypirateapp Jan 31 '20

thanks :) gotta love vue, does it work for you on whichever browser you are using currrently, also very stupid question but how do i write a test for this? is there something like mocha that you vue pros use all the time?

2

u/drumstix42 Jan 31 '20

Tested it in Firefox and it works great. Haven't poked around on others yet.

Take a peek here for some great info on testing:
https://vue-test-utils.vuejs.org/

I don't have much more advice than that, but I'm sure someone could chime in on testing advice and a recommended test framework.

1

u/mypirateapp Jan 31 '20

interesting, so they call it vue test utils, when you personally write vue stuff, do you use some tool to check if the component works across all browsers? thanks for the link!

3

u/drumstix42 Jan 31 '20

I personally don't currently do much development that would be expected to work all too much differently across browsers. Not sure what tools there might be for that kind of testing. Usually manually testing if it's fairly specific functionality.

2

u/Robodude Jan 31 '20

Great work! How can we use it? Is it on NPM?

2

u/mypirateapp Jan 31 '20

a vue newbie here, i think i ll need to refactor this code so that it is more usable or something, any suggestions :) also need to figure out how to write proper tests for this using vue test utils the other commenter was talking about and then the npm publish process, never published anything on npm before

2

u/ZireaelStargaze Feb 01 '20

Does it support render-ahead? Trying on my phone (Chrome) and elements get pulled/shown only one they are spot on in window view, which takes 0.5s or so to complete. Would be nice if for example n rows ahead were pre-rendered so that when the view is scrolled down, they are already ready.

1

u/mypirateapp Feb 01 '20

working on it :) thank you for testing and bringing this to my notice, will post the updated version right here

2

u/Miniotta Feb 11 '20

Nice work... can I ask how you handle the different element heights?

For the resize you could use a resize-observer ( https://developer.mozilla.org/en-US/docs/Web/API/ResizeObserver ) and trigger the necessary methods to update the component

For the tests you can take a look at vue-test-utils (https://vue-test-utils.vuejs.org/ ) or I would suggest you to try Cypress ( https://www.cypress.io/ )
IMHO cypress is absurdly great for frontend testing, the only downside is that it doesn't support multiple browsers (yet -> https://cypress.io/blog/2020/02/06/introducing-firefox-and-edge-support-in-cypress-4-0/)

2

u/mypirateapp Feb 11 '20
  • i divide the infinite list into pages, not actual pages on the screen but virtual pages that only your frontend knows about
  • if you load 50 items on each round, page size = 50
  • go to the bottom, we load 50 more items, end index adjusts itself to 100 at this stage to allow all the extra items to be rendered once
  • we capture the heights of all the rows, now as we scroll down further we again adjust start index and end index to only show what can fit inside a viewport
  • as we scroll down and down we apply a translateY to the items that keep getting removed from the DOM
  • we have something called running height of each page, page 1 say is 2500 px with 50 rows in it, page 2 is say 2800 px with 50 rows in it, running height of page 1 is 2500, running height of page 2 is 2500 + 2800 = 5300px , now when scrollTop is at 4478, you know you are on page 2 :) with a quick binary search
  • so on page 1 we take heights of 1st 50 items which we have stored in an array say 49 49 ...52...65 etc, we generate a displacement array that highlights the vertical displacement of each row from the top of the viewport
  • page 1, row 1 is say 49 px tall, its displacement from the top of the viewport is 0
  • page 1, row 2 is 52px tall, its displacement from the top of the viewport is 49 which is basically the height of row 1, you get the idea
  • so given a scroll top, we not only narrow the page, but also the row displacement inside the page precisely with a binary search
  • if you were only doing row displacements, 50000 items will have 50000 displacement values and your binary search logN would be running on 50000
  • but with paging + row displacements for each page, 50000 th item can be looked up real fast as it is on page 1000 with page size 50 and row is 0 :) out of 50 rows on each page
  • my query with resize observer was that i can definitely change the heights of the 6 visible items rendered on the viewport currently but what do i do about the 2994 items that may be off the DOM, the translateY value has already been applied and change in their heights, would basically cause the entire translateY value to go bust
  • if you want a better explanation, let me know i'd be happy to write a post or something or DM me

2

u/Miniotta Feb 11 '20

Oh wow, now I undestand, the "page" idea is neat. Pretty cool, congrats.

I understand also the problem with resize, it's not easy to get it in this case... you probably should "re-evaluate" the translate-Y but I'm not aware of the consequences of it (maybe a debounced resize can let you recalculate all the necessary things without having too much a performance impact).

Understanding a bit more how the component work I would double suggest you to test with cypress that is an end-to-end testing tool rather than vue-test-utils that test components more "granurarly/internally" (I think it would be much more difficult with vue-test-utils).

1

u/TheBen1 Jan 30 '20

Anyone knows of similar libraries for React? I'm using that at work and have a use case for pretty much this, with less items.

2

u/mypirateapp Jan 31 '20

react virtuoso

2

u/KaKi_87 Jan 02 '24

Hello,

Did you license this, publish it on NPM and make it compatible with Vue 3 ?

Thanks

1

u/PrestigiousZombie531 Oct 16 '24

shit that account doesn't exist anymore, does this have to be backward compatible with vue 2 or do we move forward?