Introduction
I am writing a flickr API image extractor app.
Technologies
- Nodejs with Sails framework
- Flickr API (of course)
Pre-Requisite Background
- You should have experience using Sails framework and nodejs.
- You should have a new sails project ready to put code into.
New Project Startup Steps
Loading the App Like a Process Without a Web Server on Sails
Sails is primarily a web framework, so the usual 'sails lift
' command also creates and opens a web server. For our purposes, this will not be needed, and in fact could be a security concern making the code accessible through http.
We have two options, both involve using default node command to start the app with 'node <filename>
', instead of doing sails lift on command / shell prompt. (Sidebar: I develop on windows and host on linux, so still I am going to be referring to the prompt as a command prompt or cmd)
Add the below code to the top of our app. This is one of the first things we write in our flickr-get-interesting.js file (that is what I am calling the root app code):
var Sails = require('sails');
Difference between Sails Lift / Load
Whether you use a Lift
/ Load
has to do with just one simple difference: Whether you need a web server or not. Lift
will create and start a web server.
Sails.lift({
log: {
level: 'silent'
}
}, function() {
console.log('sails started..do commands');
process.stdin.destroy();
});
OR
Sails.load({
log: {level:'verbose'}
}, function(err, sails) {
console.log('sails started..do commands');
});
We will use Load
obviously since we don't require a server, and we won't need to modify the default log level setting as well, so all we need is:
Sails.load (function(err, sails){
console.log('sails started..do commands');
});
At this point, if you execute this node flickr-get-interesting
, you should see the log we are printing.
sails started..do commands
The Flickr API
At this point, you need to have a flickr account, api_key
, etc., Which you can do here: https://www.flickr.com/services/apps/create/
Required
- flickr
api_key
from flickr App Garden - flickr secret
A Utility Class for Our Utility Methods
Let's create a flickrUtils.js file with the below code:
var util = require('util');
module.exports = {
api_key: "9d520e52f32630e1469231xxxxxxxxxx",
secret: "3e23bd445xxxxxxxx"
};
The above code will ensure we can access our api_key
/ secret with something like myUtils.api_key
if we mention this with a require on our flickr-get-interesting.js the main code file.
Initiating flickr API
flickr is one of those APIs which only needs api_key
and secret and once initiated, you query it using the API methods almost like sails blueprint API.
Let's add all our require's into our main file and initiate flickr.
Steps
- Start sails with a load - without a web server
- Initiate flickr API with the
api_key
and secret and get an object to use the flickr API methods from there on.
var util = require('util');
var Sails = require('sails');
var flickrUtils = require('./flickrUtils.js');
var oFlickr;
Sails.load(function(err, sails) {
console.log('sails started..do commands');
var Flickr = require("flickrapi"),
flickrOptions = {
api_key: flickrUtils.api_key,
secret: flickrUtils.secret
};
Flickr.tokenOnly(flickrOptions, function(error, flickr) {
oFlickr = flickr;
});
});
Before you run this, ensure all your require'd files are in the root directory on your sails project (- Remember: we are not starting a web server or creating assets to load from a different directory.. unlike a regular sails web project).
When you run it, flickr API takes quite some time to initiate.. like 2-3 minutes even (not sure, but I believe it downloads some flickr library files on first initiation... after which almost every other time, the initiation will be much quicker).
Ah! If you didn't notice: we missed a part: "how do you run it ?"
- .. not with sails lift
- use the below command on your command/shell prompt at the root directory of your sails project
node flickr-get-interesting.js
process.exit();
We did not add the process.exit();
line in our code, so flickr initiates and sails waits with a loaded app.
As we are not expecting it to run forever (we expect to run it like a cron job every few minutes), we will add the code process.exit();
Get Interesting
flickr has an 'interesting' attribute in photos and an API method to get all the recent interesting photos.
We will add a call to this, and we will move our process.exit();
into the Ajax success/error blocks of this method, so our process doesn't exit before response is received from flickr.
Flickr.tokenOnly(flickrOptions, function(error, flickr) {
oFlickr = flickr;
oFlickr.interestingness.getList({
page: 1,
per_page: 1
},
function(err, result) {
if (err) {
throw new Error(err);
finished();
}
console.log(util.inspect(result, {
showHidden: false,
depth: null
}));
finished();
});
});
});
function finished(){
process.exit();
}
In the same way, as above, you could call many different flickr methods. API reference is at https://www.flickr.com/services/api/.
The below revisions of the app, and subsequent source code versions behave different from console app.
Revision 2
What Is in This ReVision
- Refactor our code into different files like a professional project should be.
- Store flickr images in an array and process them.
Refactor
flickrUtils.js
We will write our flickr utility methods here.
process.js
Create a process.js and create all flickr API calls as wrapper methods into this.
Now to be easy to work on the flickr object from different methods, we will put the flickr object in a variable on our process object like below.
As all flickr methods are asynchronous, and to avoid writing code into the response events on flickr calls, we will create a model/object structure to store values received from flickr methods into models. You could simply use JavaScript array objects... but we are going to do this as a model in flickr.
promises
To simplify the async calls, its success and failures, we are going to use promises from the Q library.
We use Q promise library within a loop at one place which is explained here.
Revision 3
What Is in This Revision
Sails is not designed for console apps (in fact, a command 'sails run' was added and removed later in sails), and using it that way may be considered a waste of resources. Just like using PHP for console apps. Sails is moving in the direction of becoming a solid nodejs MVC framework. (In such a case, nodejs alone, as a console app is good.)
We will continue use sails, but with a web API for flickrAccess
.
- The 3rd revision of
flickrAccess
is to make a web API with a model on the backend. - The web API will be called from a browser instance and set as a cron job, just like PHP cron jobs.
- We will create an API in sails to record data to a mongo DB.
Custom Model in Sails
Sails will come with /api/models folder by default... for loading models in a structure.
We will create a custom model in this directory.
Create fPhoto.js with the below structure in /api/models.
After this, when you start sails, you will see this message.. you may choose 1 for safe, as we are not immediately using this model in connection with a database. The migrate option makes sense only in connection with a database... more here on migrate settings. And ah! yes, that does mean sails models were not designed to be used without databases (at least as of this writing on 16th May, 2015)... Also the migrate option clearly doesn't have options to be offline. So using a custom model for entity storage may be considered like pushing for using sails and you could simply use a normal array object.
Excuse my interruption, but it looks like this app does not have a project-wide "migrate" setting configured yet.
(perhaps this is the first time you're lifting it with models?)
And to avoid the above message (as it will block app execution until user entry), make the setting for 'migrate' permanently under /config/models.js.
Next Steps
We can improve on this by adding code to store values into a database.
As a Cron job
And to setup as a cron job, you would setup the below command on a cron with path to the filename:
node /myproject folder path/flickr-get-interesting.js