Readwise to Supernotes via Pipedream

Tips & Workflows
1

Just recently it came to my attention that Readwise now offers an App called Reader. Readwise has been around for several years and is a tool to manage your personal highlights discovered in text somewhere in the digital sphere. Recently, they added Reader as a fully fledged

  • Feed reader
  • Read-it-later client

beautifully combined in a single app, which really reduces the friction I experienced with Readwise before.

The fancy part is that Reader integrates the highlighting mechanism of Readwise. Plus, it offers automations for several note taking applications, such as Obsidian an Notion. Supernotes, however, is not officially supported.

End of Story?

Not at all, for where there is an API, there is a way! We can use pipedream to connect both tools using their APIs.

I created a workflow on pipedream (which is really lowcode) that creates a new note whenever I add a highlight to Readwise. This is as easy as selecting text in the Reader app, or in the Browser using the Readwise extension.

Screenshot_20230918_105538_Reader
Screenshot_20230918_105538_Reader1080×2280 146 KB

Pipedream now creates a note, which respects the syntax I traditionally use for literature notes. I can even add a personal note to the selection in Readwise, which is then also exported.

Screenshot_20230918_110405
Screenshot_20230918_1104051080×2280 182 KB

Pretty sweet, right? This even works for Youtube videos, as Readwise Reader allows for highlighting text in the transscript!

In pipedream, I had to insert a Python step to get details on the book from the Readwise API, particularly the website title and author that I use to compose the note. Feel free to ask if you have any questions on the details.

Disclaimer: I am not affiliated with either Readwise, Pipedream or Supernotes.

5 Likes
2

This is an awesome workflow @freisatz, thanks so much for sharing! :raised_hands:

Great to see how easy it is to set up and how you can customise it to your liking, looking forward to hearing how the API will open up more opportunities for you.

2 Likes
3

I stumbled upon the development docs and thought that those might be worth additional information regarding my setup.

To compose my card after a new highlight in Readwise triggers, first I gather some additional data in to consecutive python steps which I need in the card’s markup. To begin with, I make a call to the Readwise API in order to retrieve some information on the book (which contains the book or website, and the author) using this code

import requests

def handler(pd: "pipedream"):
    # Prepare inputs
    readwise_auth = pd.inputs["readwise"]["$auth"]["access_token"]
    highlight = pd.steps["trigger"]["event"]
    # Request data from Readwise API
    result = requests.get(
        f'https://readwise.io/api/v2/books/{highlight["book_id"]}',
        headers={"Authorization": f'Token {readwise_auth}'}
    )
    # Export data for use in future steps
    return result.json()

Further, I prepare some pre-formatted strings

import tld
import re
import dateutil.parser as p

def get_domain_base(url):
    dom = ""
    try:
        res = tld.get_tld(url, as_object=True)
        dom = f'{res.domain}.{res.tld}'
    except:
        dom = url
    return dom

def get_indented_text(text):
    prefix = "> "
    return re.sub("\n", f"\n{prefix}", re.sub("^", prefix, text))

def get_date(timestamp, date_format):
    d = p.parse(timestamp)
    return d.strftime(date_format)

def handler(pd: "pipedream"):
    # Set constants
    date_format = "%d.%m.%Y"

    # Prepare inputs
    book = pd.steps["book"]["$return_value"]
    highlight = pd.steps["trigger"]["event"]

    # Get domain base
    url = book["source_url"]
    domain_base = get_domain_base(url)

    # Get indented text
    text = highlight["text"]
    indented_text = get_indented_text(text)

    # Get highlight date
    book_updated = book["updated"]
    formatted_date = get_date(book_updated, date_format)

    # Return data for use in future steps
    return {
        "formatted_date": formatted_date,
        "domain_base": domain_base,
        "indented_text": indented_text
    }

Have a nice day!

3 Likes
4

Thanks for sharing your code @freisatz! We’ve updated this on the developer docs :sparkles:

5

Inspired by the request by anvanvan, I added a little script to my workflow that adds the timestamp to things that I highlight in a Youtube video’s transcript in Readwise Reader.

In case this is useful for some of you, here you go

from youtube_transcript_api import YouTubeTranscriptApi as yta
import re

def format_yt_timestamp(video_id, time):
    formatted = []
    for t in time:
        hours = int(t // 3600)
        min = int((t // 60) % 60)
        sec = int(t % 60)
        ts_url = f"{hours:02d}h{min:02d}m{sec:02d}s"
        ts = f"{hours:02d}:{min:02d}:{sec:02d}"
        url = f"https://www.youtube.com/watch?v={video_id}&t={ts_url}"
        formatted.append(f"[{ts}]({url})")

    return formatted

def normalize(str):
    str.replace("\n", ' ')
    return ' '.join(str.split())

def get_timestamps(video_id, search_word):
    transcript_list = yta.list_transcripts(video_id)
    transcript = [t.fetch() for t in transcript_list][0]

    data = [t['text'] for t in transcript]
    data.reverse()
  
    timestamp = [t['start'] for t in transcript]
    timestamp.reverse()

    text = ''
    
    time = []
    search_word = normalize(search_word)

    for i, line in enumerate(data):
        line = normalize(line)
        text = line + ' ' + text
        if search_word in text:
            time.append(timestamp[i])
            text = ''
    
    return format_yt_timestamp(video_id, time)

def handler(pd: "pipedream"):
    # Prepare inputs
    book = pd.steps["book"]["$return_value"]
    highlight = pd.steps["trigger"]["event"]

    # Get domain base
    url = book["source_url"]
    text = highlight["text"]
    video_id = ''
  
    match = re.search(r"youtube\.com\/watch.+v=([a-zA-Z0-9]{0,})", url)
    if match:
        video_id = match.group(1)

    match = re.search(r"youtu\.be\/([a-zA-Z0-9]{0,})", url)
    if match:
        video_id = match.group(1)
      
    format_timestamp =''
    if video_id:
        timestamps = get_timestamps(video_id, text)
        for ts in timestamps:
            format_timestamp = format_timestamp + "\n" + ts

    # Return data for use in future steps
    return {
        "youtube": format_timestamp,
    }

After adding a new Python step timestamp with this code, you can use the variable

{{steps.timestamp["$return_value"].youtube}}

in the markup of the final step.

3 Likes
6

In case anyone now or in the future comes back to this, I recently figured that one now needs to use a proxy in order to scrape the YouTube transcripts (which are required to automatically insert the timestamps in highlights made in YouTube videos). To make things work again, change the corresponding line to

yta.list_transcripts(video_id, proxies = {"https": "http://user:passw@server:port"})
1 Like