Skip to main content

Github Repositories Link Parser Using Bash

Hello Friends,



Welcome Again To My Blog. Today, I am going To Share With You Another Simple Bash Script That Can Parse All Repositories Links From Anyone Github Accounts Just in few Seconds. You don't need to do anything, Just Run this bash script, provide github username, Repositories types and Output filename to save all parse links. Well I'm hoping that You all are already aware of Github Repositories and Link Extracting Concept.

This Bash Script is really very to Understand and By The Way, For Completely New Bies, I had Used Good Comments Lines To Make Sure, that This Script look More Easy To Understand.

To Run This Script

chmod +x path_to_script.sh

./path_to_script.sh 
 
 
Well In This Script, I Used Various Tool To Achieve my Goals like
 
wget  = To Download Html Page Data
tr    = To Replace String Character With Another One
grep  = To Grep Matching Lines
cut   = Split Strings
uniq  = To Remove Duplicate
sort  = To Sorting
xargs = Manipulate String
 
And Bash Syntaxs. 

if You don't know that much these tools, then, don't worry use linux man page. Just Type in Your Terminal

man name_of_tool

Github Repositories Links Parser Using Bash Script


#!/bin/bash
#
# ==============================================================
#          Please, Don't Remove Author Details 
# ==============================================================
#
# Author : Suraj Singh Bisht
#          Surajsinghbisht054@gmail.com
#          www.bitforestinfo.com
#          github.com/surajsinghbisht054
#
#
#
# Description:
#            This Script Can Extract All Available Repositories Links With Details From
#            Anyone Github Account.
#
#
#



# instruction to use script

# usages details
if [ $# = 1 ]; then
 if [ $1 = "--help" ] || [ $1 = "-h" ]; then
  printf "====================================================================\n";
  printf "\--------------------- This is Help Menu ---------------------------/\n";
  printf "====================================================================\n";
  printf "                                      Author: Suraj Singh \n";
  printf "                                      surajsinghbisht054@gmail.com\n\n";
  printf "\---------------------- Read Me -----------------------------------/\n";
  printf "       This program can extract Github repositries links of Any \n";
  printf "  Specific Account. If you want to find all repo links of anyone Github,\n";
         printf "  Just Run This parse over that Account. This script can extract links\n";
         printf "  and Can Save All Links into txt files.\n\n";
  printf "Simple Usages:\n";
  printf "\n\tProgram Usages : %s [url] [repo_type] [output]\n\n" $0;
  printf "Url : https://github.com/example_user_name/\n";
  printf "Repo : fork, source\n";
  printf "output : file_name_to_save Output\n";
  printf "\n\nExample\n"
  printf "  %s github_user_name source links.txt\n\n\n" $0;
  exit;
 fi
else
 if [ $# != 3 ]; then
       printf "  %s --help\n" $0;
         printf "\nProgram Usages : %s [url] [repo_type] [output]\n\n" $0;
  exit;

 fi;
fi;


# initialize user inputs
USERNAME=$1;
REPO_TAB=$2;
OUTPUT_F=$3;

# Time To Download First Page of Account
w="https://github.com/$USERNAME?tab=repositories&type=$REPO_TAB";

printf "\n\nGenerating Github link : %s\n" $w

wget $w -O /tmp/github_link_extractor

for((n=0; n<1; n++))
do

 cat /tmp/github_link_extractor | tr ' ' '\n' | tr '>' '\n' | grep -i "href=[^>]" | cut -d '"' -f 2 | uniq | sort | grep -e "/$USERNAME/.*" | xargs -n 1 -I% echo "https://github.com%" > $OUTPUT_F ;

 token=$(cat /tmp/github_link_extractor | tr " " "\n" |grep -e "href="| tr ">" "\n" | grep -e "after" | cut -d"=" -f3 | cut -d"&" -f 1);
done;


for((n=0; n<1; n++))
do
 if [ ${#token} -le 90 ];
 then
  w="https://github.com/$USERNAME?after=$token&tab=repositories&type=$REPO_TAB";
  printf "\n\nGenerating Github link : %s\n" $w
  wget $w -O /tmp/github_link_extractor
  cat /tmp/github_link_extractor | tr ' ' '\n' | tr '>' '\n' | grep -i "href=[^>]" | cut -d '"' -f 2 | uniq | sort | grep -e "/$USERNAME/.*" | xargs -n 1 -I% echo "https://github.com%" >> $OUTPUT_F ;

         token=$(cat /tmp/github_link_extractor | tr " " "\n" |grep -e "href="| tr ">" "\n" | grep -e "after" | cut -d"=" -f3 | cut -d"&" -f 1);

  echo $token;
  echo $((n--));
 fi;

done

Usages :


./Github_Link_Parser.sh github_user_name source links.txt


For More Updated Example, Check Github Here

 

Related Post

Top Visited

how to install burp suite in Linux/Ubuntu 16.04

Create Simple Packet Sniffer Using Python

Big List Of Google Dorks For Sqli Injection

List of Keyboard Shortcuts Keys for GNOME Desktop (Kali linux / Linux / Ubuntu/*nix )

Latest Google Dorks List

How to create Phishing Page Using Kali Linux | Webpage Page Cloning Using Kali Linux Social Engineering Toolkit

how to configure burpsuite with firefox?

What is the use of Pseudo header in TCP/UDP packets?

Python Beautiful Soup Module - Tutorial - Part 2

Create Ping Sweeping Script Using Python