A simple Pinboard API example

I wrote up an example using the Pinboard API from Node: this is written with the idea that I might combine it with other external services (dropbox, facebook, etc).

The ideal situation here is to iterate over recently bookmarked posts, then call some function on each unseen result (external API, log to screen, etc), like so:

get_posts(pinboard_api, 'recent', pinboard_user_data)
  .then( 
    (data) => {
      async.map(data, do_something)
    }
  ).error(
    err
  )

This ended up being a pretty fascinating trip down the vast list of node modules, and I ended up with this:

let _ = require('lodash'), 
    request = require('request'),
    Promise = require('bluebird'),
    rp = require('request-promise'),
    xml2js = require('xml2js'),
    async = require('async');

lodash is a nice functional programming library – great utility belt API for working on collections
bluebird is a promise library. I don’t really like promises, but they do let you flatten some of the nesting that shows up in Node callbacks. There is another library, “Q”, that seems to be more popular, but Bluebird promises better error handling.
request seems to be the most popular way to do curl/ajax/wget style calls
request-promise lets you use HTTP request results as promises
xml2js lets you parse XML text into JS objects (Pinboard returns XML)
async is a library like lodash, but lets you turn the “map” type functions into parallel loops

Next, I’ve defined some structure about the API, thinking I might want to support other APIs. Here are the important facts about this one:

let pinboard_api = {
  'auth': 'auth_token',
  'recent': {
    returns: ["href","time","description","extended","tag","hash","shared"],
    url: 'https://api.pinboard.in/v1/posts/recent',
    unique_key: 'hash',
    implicit_keys: {
      count: 100
    },
    iter: (result) => result.posts.post  
  }
}

Authentication is usually the trickiest bit with these integrations (aside from SSL). Pinboard just takes a single authentication token, on your account:

let pinboard_user_data = {
  'auth_token': '-------------',
}

Now, to actually do the work we can combine all this together. Conceptually it’s really simple but complexified a bit by my attempts to leave room to support asynchronous operations + other endpoints and APIs (note I haven’t implemented hiding things that have been seen recently – imagine that is a lookup elsewhere).

function post_fields(api, method, user_data) {
  let qs = {};
  qs[api.auth] = user_data[api.auth];  
  _.extend(qs, api[method].implicit_keys);
  
  let options = {
    uri: api[method].url,
    qs: qs,
    headers: {
      'User-Agent': 'Request-Promise'
    } 
  };
      
  return new Promise(
    (resolve, reject) => {
      rp(options)
        .then(
          (resp) => {
            xml2js.parseString(resp, 
              (err, result) => {
                if (!result) {
                  reject(err); 
                }                  
                
                resolve(
                  _(api[method].iter(result))
                    .filter(
                      (entry) => !seen(entry)
                    )
                    .map(
                      (entry) => {
                        markSeen(entry.$)
                        
                        return entry.$;
                      }  
                    ).value()
                )
              });
          }
        )
        .error(reject)    
    }    
  );
}