Graph view with 2000+ notes

I currently have 2375 notes in Supernotes, which slows the graph view rendering down to a crawl (I’d guess less than 5 fps in 2D, and around 1 fps in 3D). This is on a 2020 MacBook Pro M1. Is there any chance to optimise the rendering so this feature is usable for people with larger databases?

Also, I’m disappointed with how the clustering algorithm is currently configured. I was hoping to see clearer groupings of related notes, not just a tangled mess of notes:

Are you using a spring function on the edges to pull the connected nodes together? If so, I have a theory that you might want to reduce the weight of the springs for connections that cross root-nodes boundaries. That might keep the root-node clusters more clearly defined, and create more logical clusterings.

We definitely want to improve the performance of the graph view. There are actually some optimizations we were wanting to add before release but they resulted in wonky behavior so didn’t ship with the initial 2.0 version, but hopefully we will figure out those issues soon and can get them in.

We have actually seen better performance with 2000+ node graphs than you are experiencing, so one thing I’d like to double-check is that you are using the ARM-optimized build of the Supernotes desktop application, because that is important to maximize perf. But you also have data that is probably more heavily linked than what we were stress-testing on so that might be part of the reason you’re seeing such poor performance (assuming you’re using the ARM build).

Thanks for bringing our attention to clustering issues in your graph, clustering is something we haven’t put very much time into yet, but is another aspect of the graph view we’d like to continuously improve. Good thinking on the spring function, we will see if that works.

1 Like

Hi Connor,

I’d totally missed the ARM selector on the download page, so I’ve been running the Intel version. The ARM-optimised build has definitely improved the graph view performance a little, at least on the 2D view. I’d guess the framerate is above 10 now, which is slightly more usable. The fps on the 3D view doesn’t seem to have improved.

I’ll be following the upcoming performance and clustering improvements with great interest!

Enjoy your weekend, and congrats on the launch of 2.0!

I’m on an Intel MacBook Pro and the 3D graph actually renders quickly. I can move around it / rotate it without any speed issues. I also have over 2000 cards. The fan does heat up sometimes though.

That’s interesting. I wonder if it’s because you have more GPU horsepower, or because our databases have a different number of connections, or both.

In my hierarchy I have about 100 parent cards (that have child cards), and maybe 400 backlinks. A number of cards have two or more parents. A fair number of cards are color coded. With all 2000-plus cards with these connections in 3D (or 2D) graph view, I have no speed complaints, just the occasional fan noise.

As my notes database has grown in size (I’m up to 3762 notes now), my challenges with the rendering speed and clustering of the graph view have become worse:

Rendering

Supernotes has a target interaction speed of <100ms. I hope that my screen recording clearly demonstrates how far away from that goal the current graph view is on an M1 MacBook Pro.

BTW, I click to navigate to the Insights note at ~29 seconds into the video, but the UI doesn’t start responding until ~32 seconds into the video. (You’ll also notice the double rendering / re-rendering bug when the Insights graph does start rendering).

Clustering

I think the main problem with your clustering algorithm is that your edge springs appear to give equal weight to “friend” links as they do to “siblings”. In a highly connected database like mine, that leads to essentially random placement of nodes in the graph. That’s neither beautiful or useful.

I’d suggest changing this so “sibling” and “parent” links get a much stronger attraction factor, with “friend” links being significantly weaker. This should let the natural structure of the tree be visible in the graph, whilst still pulling related branches closer to each other (via the weaker attraction of the “friend” links).

Hi @JamesT,

We hear you! We’ve been dabbling with the Graph View behind the scenes to try to eek out performance. The reason that both your rendering and clustering is not optimal is that as you’ve mentioned you have an extremely interlinked (thousands of links as well as cards) database, and that mostly likely explains why @JohnCP above isn’t having the same issues as you with only 400 backlinks.

The biggest performance drain is the animation at the start, we can remove this and greatly improve performance but it will feel less fluid transitioning when dragging nodes from one place to another. We can also definitely make some big improvements to the clustering in future updates, and that’s a great suggestion about weighting more on the parent links than bi-directional card links.

We understand your frustrations and it’s very much our goal to get all interactions under <100ms but with huge graphs there is only so much we can optimise. As you’ve noticed with your Insights parent card, breaking down and viewing portions of your graph within a parent + view depth is the best way to visualise your graph at the moment. This is on the forefront of our minds and we look forward to pushing updates to improve this soon.

Hi @Tobias,

Thank you for your detailed response.

It’s interesting to discover that you consider my database “huge” and that you’re not optimising performance for databases of that size/connectivity. That’s new information to me, which definitely affects my long-term thinking around my choice of Supernotes.

I had assumed that my use case (a life-long digital Zettelkasten) would be well within your goals for the tool. If it’s not, then I should start looking elsewhere, where that use case is part of their long-term plans.

A sad day for me.

Hi @JamesT,

That may have came across wrong, your database isn’t huge, you’re correct in thinking that, we can support a lot larger databases don’t worry :wink:

Just visualising a graph with that many nodes is considered from a developer point of view as large, even the library we use classes this graph as large :joy: and it’s a lot smaller than yours.

So hopefully that clears things up and it’s not a sad day. We’ve got many plans in the works to improve the rendering and clustering of graphs and we can’t wait to share them with you.

That makes sense. Thank you for clearing up my misconception.

I probably just need to recalibrate my assumptions about where and how I’ll be able to use the graph view. Like you mentioned, using it on sub-sections of the graph already works well, so I can learn to live with that.

:heart:

1 Like

That’s ok, that’s what we are here for :heart:

We do take all your suggestions on board, and discuss and record them internally all the time, even the Cosmograph library you suggested a while back! It’s faster since it uses WebGL and we use Canvas – however switching is a large task and there are other drawbacks like less compatibility (it didn’t work on iOS for example).

A lot happens behind the scenes, the challenge recently was adding labels to graphs without drastically reducing performance. Looking forward to sharing more with you!.

1 Like

2 posts were split to a new topic: Graph View re-renders too much