How to save blogger's json feed to the server for widget development

Description:

savecbf(Save category based feed) is Blogger label based json feed collector.
Main purpose of this project is saving json data by category in the server to use it for blogger widgets(gadgets).

Feature

This script works for more than 150 posts feed. It labels the file name by 4 digits(eg. 0000) index numbering. Which mean 0001 will be first 150 posts data and 0002 will be 151~300 posts feed data. This script generates a directory itself to store json files. Which means you don't have to worry about creating a directory for json files. This script supports other languages too.

Requirements

1. Web server: There's nothing free! At least this script is free.
2. PHP in Web server: Only testing and debugging in 7.1xx therefore it is better to be the same version however it wouldn't really make a problem with other versions.

Setting up

1. Change blog address in savecbf.php file.
2. Upload savecbf.php to web server.
3. Use task scheduler(eg. crontab in linux) to run the script hourly or daily.

Problems & Solve

1. Why not using database?:
Files are quicker, database is slower. JSON files are good enough for blog data. However it can be great to use database.

2. The name of JSON file is ugly.. why does it have to use urlencode?
To support other languages in any OS. I haven't done testing for linux yet but it should be working on linux because i have used similar script on debian server and it only required small changes.

3. Why PHP? PHP is so bad..
You can build same script with other language.

License

MIT, Simply just use it. I have built it for blogger widgets(gadgets).

Source Code


<?php
// Version: 1.0
// Get category(label) data from main url
// Your blog address, example:
// https://example.blogspot.com/
// then $blogAddr = example;
$blogAddr = 'example';
$dirPath = './' . $blogAddr . 'RssJson';
// if there is no directory for json files then create one, otherwise just skip the creating directory.
if (!file_exists($dirPath)) {
    mkdir($dirPath, 0755, true);
}
$mainRssUrl = 'http://' . $blogAddr . '.blogspot.com/feeds/posts/default?alt=json&&max-results=0';
$mainJson = file_get_contents($mainRssUrl);
// we decode json to dimentional arrays
$mainJsonArry = json_decode($mainJson, true);
// Go through json data by category and save them as ae file
foreach($mainJsonArry['feed']['category'] as $categories => $category){
    $urlEncodedCategory = urlencode($category["term"]);
    $categoryRssUrl = 'http://' . $blogAddr . '.blogspot.com/feeds/posts/default/-/' . $urlEncodedCategory . '?alt=json&start-index=1&max-results=150';
    $categoryJson = file_get_contents($categoryRssUrl);
    $categoryJsonArry = json_decode($categoryJson, true);
    // if the category contain more than 150 posts, we go through whole posts that is more than 150
    if($categoryJsonArry['feed']['openSearch$totalResults']['$t'] > 150){
      $totalNumberOfPost = $categoryJsonArry['feed']['openSearch$totalResults']['$t'];
      $numberOfFiles = $totalNumberOfPost / 150;
      if(($totalNumberOfPost / 150) > (floor($totalNumberOfPost / 150))){
        $numberOfFiles = $numberOfFiles + 1;
      }
      for($i = 1; $i <= $numberOfFiles; $i++){
        $maxResult = $i * 150;
        $startIndex = 1;
        if($i > 1){
          $startIndex = 1 + (($i - 1) * 150);
        }
        $categoryRssUrl = 'http://' . $blogAddr . '.blogspot.com/feeds/posts/default/-/' . $urlEncodedCategory . '?alt=json&start-index=' . $startIndex . '&max-results=' . $maxResult;
        $categoryJson = file_get_contents($categoryRssUrl);
        $categoryJsonArry = json_decode($categoryJson, true);
        $fileIndex = sprintf("%04s", $i);
        file_put_contents($dirPath . '/' . $urlEncodedCategory . '_' . $fileIndex . '.json', $categoryJson);
      }
    }
    // number of total post is lower 150 therefore we do not have to do anything complicated.
    // simply save the one we have.
    else{
      file_put_contents($dirPath . '/' . $urlEncodedCategory . '_0001.json', $categoryJson);
    }
};
?>


GitHub: https://github.com/treezi1004/saveCategoryBasedFeed

No comments:

Post a Comment