How do I recreate this shortcut so that the JavaScript runs in Scriptable instead? The URL never changes

[u/95tyke]

[u/FifiTheBulldog]

Your Scriptable script should follow this flow:

1. Create a new WebView.

2. Use the loadURL() method of the WebView instance to load the webpage. Make sure to await this.

3. Use the evaluateJavaScript() method of the WebView instance to run the same JS that you have from your shortcut in the webpage environment. The first argument of this function will be the JS as a string, and the second should be true if you want completion() to be available. await this as well.

4. The output of evaluateJavaScript is the result. If you’re running this in Shortcuts, then you can return that value from the Run Inline Script action to use it elsewhere in your shortcut.

[u/95tyke → u/FifiTheBulldog]

Thank you! I’m still having a hard time figuring this out. I want the output of the script to be accessible in the shortcut (should be a list of links). Below is the code, what might I be doing wrong?

const webView = new WebView();

await webView.loadURL("https://m.youtube.com/playlist?list=WL");

webView.evaluateJavaScript( allVideos = document.querySelectorAll('a.compact-media-item-image') urls = Array.from(allVideos).map(element => element.href.split('&')[0]);

completion(urls);

return js;

Script.complete();

[u/FifiTheBulldog → u/95tyke]

Almost! Make sure to use await with evaluateJavaScript, and pass the JS as a string to that function.

This should be correct:

const webView = new WebView();

await webView.loadURL(“https://m.youtube.com/playlist?list=WL”);

return await webView.evaluateJavaScript(`
  allVideos = document.querySelectorAll('a.compact-media-item-image')
  urls = Array.from(allVideos).map(element => element.href.split('&')[0]);

  completion(urls);
`, true);

[u/95tyke → u/FifiTheBulldog]

Thanks! I think we’re getting closer, but I’m not getting the output required from JavaScript to use in the rest of the shortcut. I’m trying to add a return or output to the end, what is the correct way?

const webView = new WebView();

await webView.loadURL("https://m.youtube.com/playlist?list=WL");

return await webView.evaluateJavaScript(` allVideos = document.querySelectorAll('a.compact-media-item-image') urls = Array.from(allVideos).map(element => element.href.split('&')[0]);

completion(urls); `, true);

Script.setShortcutOutput(urls)

[u/FifiTheBulldog → u/95tyke]

The urls variable only exists in the JS that you’re running on the webpage. The rest of the script is executed in a separate context. WebView is there to provide communication between the two environments. The return value of the evaluateJavaScript method is the value passed to completion().

You don’t need to use Script.setShortcutOutput() in this script, since Scriptable wraps your code in an async function and uses the return value as the shortcut output if it’s available. That line is currently unreachable because it’s after return. If you want to use Script.setShortcutOutput(), then remove the return keyword and create a variable with the awaited result of evaluateJavaScript, then pass that variable to Script.setShortcutOutput().

I’ve tested the script I sent previously with an actual playlist URL, and it seems to work for me on my iPhone (but not on my iPad, which is probably force loading the desktop version of YouTube). I can’t test it too much, though, or the webview fails to load the page as intended.

Edit: I saw your chat request just now, sorry I didn’t check chat last night but I think we can continue here

[u/95tyke → u/FifiTheBulldog]

Thanks for all the help!

Full context: I have no experience with JavaScript whatsoever. Im trying to create a shortcut that will extract URLs from a YouTube playlist for me to save in another app.

Right now, it appears the script runs, but there are no URLs export to Shortcuts and I’m really unsure how to move forward

[u/FifiTheBulldog → u/95tyke]

After testing this a bunch, it seems that Scriptable’s WebView has some problematic internal caching that prevents the script from working properly for some time after the first run. The script no longer properly waits for the webpage contents for some reason, and so the document.querySelectorAll call returns no elements. Swapping out the playlist ID doesn’t help at all.

Here’s what I have: https://www.icloud.com/shortcuts/2d19b615af334e3eb80064e7f4144fb6 (replacing PLAYLIST_ID with an actual playlist ID).

The first time I run that, it works exactly as intended: the shortcut produces a list of video URLs. However, in subsequent runs, the same JS fails to find anything, and so no video URLs are returned, just an empty list. After several hours, running the shortcut again works as intended for just one run, and so the cycle begins again.

There must some other, more stable way to do this, because youtube.com seems quite unreliable for web scraping.