=================
== Claus' Blog ==
=================
Made with Hugo and ❤️

Writing a Hugo Raindrop.io bookmark integration with AI

#python #bookmarks #howto #blog #personal

Table of Content

bacardi55 [1] has an interesting bookmarks section on his blog, where a script generates Hugo markdown pages for bookmarks he has added to a central (self-hosted) bookmark manager. I found this very interesting and wanted to build something similar. I’ve been using raindrop.io for some time now, and this service also provides an API. So this could be an easy project to do with the help of Github Copilot, which I have recently been using to give my software development projects a boost.

So here are my goals:

  1. Generate Hugo Markdown content using the Raindrop.io API (link and note)
  2. Use only bookmarks from a specific publicly shared collection
  3. A new section on my blog, only for bookmarks
  4. Use python
  5. Use Github Copilot for creating the python script
  6. Find a way to automate the update of the GitHub pages

There was one step in the workflow that I was not sure how to do: I don’t want to manually deploy my pages every time I bookmark a page. Ideally, saving a bookmark would execute a web-hook, but Raindrop.io does not support this. So the alternative is to run a script on my server that fetches new bookmarks, generates the markdown files and pushes them to Github pages. But for now I will do it the manual way and decide later how improve it.

The AI guided script generation

I used the following prompt to generate the Python script with GitHub Copilot. I just selected the following snippet of text and used the Visual Code Copilot plugin: “Write the Python script based on my comments in file”:

 1# import API token, API endpoint and collection ID from .env file
 2# RAINDROP_API_KEY=
 3# RAINDROP_COLLECTION_ID=46414428
 4# RAINDROP_ENDPOINT=https://api.raindrop.io/rest/v1/raindrops/
 5
 6# get the last 5 bookmarks from the API endpoint
 7# use only links with a note
 8
 9# generate markdown files with the following content
10# +++
11# title = <title>
12# date = <bookmark creation date>
13# categories = [ 'bookmarks' ]
14# +++
15# 
16# <note>
17# 
18# <link>
19# <excerpt>
20
21# generated files stay in subfolder content/bookmarks
22# don't process links you already have a markdown file for

The result was almost usable on the first try. Only the variable for the endpoint and the handling of the API date field (which is a string, but Copilot assumed an integer) needed manual fixing. Here is the output:

 1import os
 2import requests
 3from datetime import datetime
 4from dotenv import load_dotenv
 5
 6# Load environment variables from .env file
 7load_dotenv()
 8api_key = os.environ.get('RAINDROP_API_KEY')
 9collection_id = os.environ.get('RAINDROP_COLLECTION_ID')
10endpoint = os.environ.get('RAINDROP_ENDPOINT')
11
12# Check if the required environment variables are set
13if not api_key or not collection_id or not endpoint:
14    print("Please set the environment variables")
15    exit(1)
16
17# Get the last 5 bookmarks from the API endpoint
18url = f"{endpoint}/{collection_id}?perpage=5"
19headers = {"Authorization": f"Bearer {api_key}"}
20response = requests.get(url, headers=headers)
21
22# Check if the request was successful
23if response.status_code != 200:
24    print("Failed to retrieve bookmarks from the API")
25    exit(1)
26
27# Process the bookmarks
28bookmarks = response.json().get("items", [])
29for bookmark in bookmarks:
30    link = bookmark.get("link")
31    excerpt = bookmark.get("excerpt")
32    note = bookmark.get("note")
33    tags = bookmark.get("tags")
34    created_timestamp = int(datetime.fromisoformat(bookmark.get("created")).timestamp())
35    creation_date = datetime.fromtimestamp(created_timestamp)
36
37    # Generate the markdown file path
38    file_name = f"content/bookmarks/{creation_date.strftime('%Y%m%d')}.md"
39
40    # Check if the markdown file already exists
41    if os.path.exists(file_name):
42        continue
43
44    # Generate the markdown content
45    markdown_content = f"""+++
46title = "{bookmark.get('title')}"
47date = "{creation_date}"
48categories = [ 'bookmarks' ]
49tags = {tags}
50+++
51
52{note}
53
54{link}
55
56{excerpt}
57"""
58
59    # Write the markdown content to the file
60    with open(file_name, "w") as file:
61        file.write(markdown_content)
62
63print("Bookmark generation completed.")

The Theme - The hardest part

I wasn’t expecting that: Customising the theme was the part that took the most time. Not that it was complicated or anything. A change here, a change there… Made me understand the core of Hugo a bit more. So I’m ok. But for this blog post it’s too much to point out all the changes. In a nutshell, that’s what I did:

  1. I added a section named content/bookmarks.
  2. I added a menu item in my config.toml
  3. I added a small section on the mainpage for the last 5 bookmarks. For that I added this code:
 1<h3>🔗 Recent bookmarks</h3>
 2<ul>
 3    {{ range first 5 (where site.RegularPages "Section" "bookmarks") }}
 4    <li>
 5        <div class="post-title">
 6            <time>{{ .Date.Format "2006-01-02" }}</time> 
 7            <a href="{{ .RelPermalink }}">{{ .Title }}</a>
 8    </li>
 9    </div>
10    {{ end }}
11</ul>

Automation

I thought: “How can I make this automatic, but as simple as possible?” Since my deployment is through GitHub pages, this means that I always have to push my Markdown files to my GitHub repository. This also means I have to use the git client and do a pull/push for each bookmark update.

So I need a computer to fetch my repository, run the python script on the local repository, git commit/push the changes to GitHub. To do this, I also need access to GitHub via a password-less SSH deployment key [2]. A deployment key can be explicitly added to a single repository and only allow access to that repository.

  1. Adding my python script to the repository

  2. Generate a ssh-key (w/o password): ssh-keygen -t ed25519 -C "your_email@example.com" ./deploy_key and added the public key to the Deployment section of my repository

  3. Added an entry to my ~/.ssh/config to use the new ssh key for that GitHub repository:

    1Host blog
2    Hostname github.com
3    IdentityFile ~/.ssh/git_deploy_key
4    IdentitiesOnly yes 
5    AddKeysToAgent yes

  4. Clone the repository only for the bookmark creation workflow: git clone git@blog:cloonix/cloonix.github.io ./bookmark-update

  5. Running a script through crontab:

    1git pull
2python3 raindrop_bookmarks.py
3git add ./content/bookmarks
4git commit -m "updated raindrop bookmarks"
5git push

  6. Created a crontab to run the script.

Conclusion

This is not a step-by-step howto. More a diary of that kinda simple integration of the Raindrop API into my Hugo Github Pages deployment. But i achieved my goal: The crontab is running every how, which is more than enough, and fetches new bookmarks from my public bookmark collection. It generates the Markdown files and pushes them to Github where Actions deploy the static site.

[1] https://bacardi55.io/2024/02/13/bookmarks-section-the-pesos-way-kind-of/
[2] https://docs.github.com/en/authentication/connecting-to-github-with-ssh/managing-deploy-keys