Working with the Clarifai API

Clarifai

As another example of how R can interact with APIs, we’ll look at the Clarifai API. Clarifai is an NYC-based company that offers image recognition solutions using artificial intelligence. One of their products is an API that will apply a variety of machine learning models to detect the objects in a picture or video. You can see an example of it can do here.

Although there is an excellent R package to interact with the Clarifai API, created by Gaurav Sood, unfortunately at the moment it doesn’t work with the current authentication system, so we’ll write our code from scratch.

We’ll be replicating the example here.

The first step is to load the package we will use to make the requests, httr and add the base URL for this API endpoint, as well as the API key (we will use mine for now, but you can create yours here).

library(httr)
apikey <- 'c6f237cf41ae4f3687acc253e23d0a46'
base_url <- "https://api.clarifai.com/v2/models/aaa03c23b3724a16a56b629203edc62c/outputs"

Now we will add the body of our request – the URL of the image we want to classify (this one), embedded within an object in JSON format and with a specific structure. (See the website above for more details.)

We can either type the object in JSON as text (recommended in this case, given its complexity), or create it from within R as a list:

requests <- '
  {
    "inputs": [
      {
        "data": {
          "image": {
            "url": "http://i.imgur.com/XmAr3jV.jpg"
          }
        }
      }
    ]
  }'

req <- list("inputs" = list())
req$inputs[[1]] <- list(data=list(image=list(url = "http://i.imgur.com/XmAr3jV.jpg")))
requests <- rjson::toJSON(req)

And now we’re ready to run the query! Note that unlike the previous examples, since here we’re sending some additional data, we need to do a POST request instead of a GET:

r <- POST(base_url, 
    add_headers(
        "Authorization" = "Key c6f237cf41ae4f3687acc253e23d0a46",
        "Content-Type" = "application/json"),
    body = requests)
r

## Response [https://api.clarifai.com/v2/models/aaa03c23b3724a16a56b629203edc62c/outputs]
##   Date: 2018-07-30 14:42
##   Status: 200
##   Content-Type: application/json; charset=UTF-8
##   Size: 2.2 kB

Did it work? So far so good. Now let’s try to parse the response into a list

r <- content(r, "parsed")

We can extract the parts of the list that we need to see the main labels predicted for this picture:

for (result in r$outputs[[1]]$data$concepts){
    message('object: ', result$name, ' -- probability: ',
        result$value)
}

## object: dog -- probability: 0.99959356

## object: canine -- probability: 0.9978343

## object: pet -- probability: 0.99582434

## object: puppy -- probability: 0.9927995

## object: cute -- probability: 0.99143314

## object: mammal -- probability: 0.98917997

## object: friendship -- probability: 0.9766187

## object: portrait -- probability: 0.97558284

## object: domestic -- probability: 0.96942294

## object: little -- probability: 0.9688714

## object: animal -- probability: 0.9677278

## object: sit -- probability: 0.964401

## object: breed -- probability: 0.95978403

## object: adorable -- probability: 0.9550632

## object: whelp -- probability: 0.9442301

## object: pedigree -- probability: 0.9439126

## object: studio -- probability: 0.94348264

## object: retriever -- probability: 0.9359648

## object: purebred -- probability: 0.93474025

## object: no person -- probability: 0.91294944