In this article, you will see my very first self-written Golang code to write a tool that will be started from a Jenkin’s job by a cron and will check an organization’s public repositories list in the Github.
Introduction
The task is to write a tool which will be started from a Jenkin’s job by a cron and will check an organization’s public repositories list in the Github.
Then it has to compare received repositories list with a predefined list with allowed repositories and if they will be not equal – send an alert to a Slack channel.
The idea is to have such a check in case developers will accidentally create a public repository instead of private or will change a private repository to the public and get a notification about such issue.
It could be written with bash and curl, or Python with urllib and use Github API directly, but I want to have some Go experience so will use it.
In fact – this is my very first self-written Golang code which I didn’t use a lot and even don’t know its syntax, but will try to use some existing knowledge with C/Python, add some logic and of course with the Google’s help and write something workable.
To work with API Github, the go-github package will be used.
Let’s begin.
Add imports, the function was created automatically with the vim-go plugin:
package main
import (
"fmt"
"github.com/google/go-github/github"
)
func main() {
fmt.Println("vim-go")
}
Add $GOPATH:
$ sudo mkdir /usr/local/go
$ sudo chown setevoy:setevoy /usr/local/go/
$ export GOPATH=/usr/local/go && export GOBIN=/usr/local/go/bin
Install package:
$ go get
_/home/setevoy/Scripts/Go/GTHB
./gitrepos.go:5:2: imported and not used: "github.com/google/go-github/github"
Getting Repositories List From Github
Copy and paste the first example from the package’s README, an organization name will be taken from the $GITHUB_ORG_NAMEand let’s try to get something.
Set the variable:
$ export GITHUB_ORG_NAME="rtfmorg"
And code:
package main
import (
"context"
"fmt"
"github.com/google/go-github/github"
)
func main() {
client := github.NewClient(nil)
opt := &github.RepositoryListByOrgOptions{Type: "public"}
repos, _, _ := client.Repositories.ListByOrg(context.Background(),
os.Getenv("GITHUB_ORG_NAME"), opt)
fmt.Printf(repos)
}
In the original example, organization name was hardcoded but here we will take it from a variable which will be set in a Jenkin’s job using os.Getenv.
Add context and os imports.
Run the code:
$ go run go-github-public-repos-checker.go
command-line-arguments
./go-github-public-repos-checker.go:17:15:
cannot use repos (type []*github.Repository) as type string in argument to fmt.Printf
Errr…
Okay.
I thought client.Repositories.ListByOrg will return actually list – just because of the List in its name. 
Check what data type we have in the repos object. Use reflect:
package main
import (
"os"
"context"
"fmt"
"reflect"
"github.com/google/go-github/github"
)
func main() {
client := github.NewClient(nil)
opt := &github.RepositoryListByOrgOptions{Type: "public"}
repos, _, _ := client.Repositories.ListByOrg(context.Background(),
os.Getenv("GITHUB_ORG_NAME"), opt)
fmt.Println(reflect.TypeOf(repos).String())
}
Run:
$ go run go-github-public-repos-checker.go
[]*github.Repository
Good… It’s really list ([]), apparently pointed to a structure – github.Repository.
Let’s check with the go doc:
go doc github.Repository
type Repository struct {
ID *int64 `json:"id,omitempty"`
NodeID *string `json:"node_id,omitempty"`
Owner *User `json:"owner,omitempty"`
Name *string `json:"name,omitempty"`
FullName *string `json:"full_name,omitempty"`
Description *string `json:"description,omitempty"`
...
Yup – it’s structure and also we see all its fields.
To disable the “imported and not used: “reflect” message – set it as _reflect:
package main
import (
"os"
"context"
"fmt"
_"reflect"
"github.com/google/go-github/github"
)
func main() {
client := github.NewClient(nil)
opt := &github.RepositoryListByOrgOptions{Type: "public"}
repos, _, _ := client.Repositories.ListByOrg(context.Background(),
os.Getenv("GITHUB_ORG_NAME"), opt)
for _, repo := range repos {
fmt.Printf("Repo: %s\n", *repo.Name)
}
}
Here from the repos list, we are getting an element’s ID and its value.
Can add IDs as well:
...
for id, repo := range repos {
fmt.Printf("%d Repo: %s\n", id, *repo.Name)
}
...
$ go run go-github-public-repos-checker.go
0 Repo: org-repo-1-pub
1 Repo: org-repo-2-pub
Comparing Lists in Go
Now need to add another one list which will keep repositories list from Github.
Allowed to be public repositories will be passed from Jenkins in a list view separated by spaces:
$ export ALLOWED_REPOS="1 2"
Attempt Number One
Create allowedRepos of the string type and save allowed public repositories list in it:
...
allowedRepos := []string{os.Getenv("ALLOWED_REPOS")}
for id, repo := range repos {
fmt.Printf("%d Repo: %s\n", id, *repo.Name)
}
fmt.Printf("Allowed repos: %s\n", allowedRepos)
...
But here is an issue which I faced a bit later when I started comparing lists. We will see it shortly.
Run the code:
$ go run go-github-public-repos-checker.go
0 Repo: org-repo-1-pub
1 Repo: org-repo-2-pub
Allowed repos: [1 2]
Okay – it works.
Now, we need to make checks between two repositories lists – repos with repositories from Github and allowedRepos.
The first solution was next:
...
allowedRepos := []string{os.Getenv("ALLOWED_REPOS")}
fmt.Printf("Allowed repos: %s\n", allowedRepos)
for id, repo := range repos {
for _, i := range allowedRepos {
if i == *repo.Name {
fmt.Printf("Index: %d, repo %s found in Allowed as %s\n", id, *repo.Name, i)
} else {
fmt.Printf("ALARM: repo %s was NOT found in Allowed!\n", *repo.Name)
}
}
}
...
Run:
$ go run go-github-public-repos-checker.go
Allowed repos: [1 2]
ALARM: repo org-repo-1-pub was NOT found in Allowed!
ALARM: repo org-repo-2-pub was NOT found in Allowed!
Looks like it works?
Repositories with the 1 and 2 names != org-repo-1-pub and org-repo-2-pub.
But the problem which might be already obvious for some readers appeared when I did a “back-check”, i.e., when I set $ALLOWED_REPOS with real names to get the OK result:
$ export ALLOWED_REPOS="org-repo-1-pub org-repo-2-pub"
Check:
$ go run go-github-public-repos-checker.go
Allowed repos: [org-repo-1-pub org-repo-2-pub]
ALARM: repo org-repo-1-pub was NOT found in Allowed!
ALARM: repo org-repo-2-pub was NOT found in Allowed!
And Why?
Because in the allowedRepos, we have not a list aka slice and Go – but just a string.
Let’s add the i variable’s output and indexes numbers:
...
for r_id, repo := range repos {
for a_id, i := range allowedRepos {
fmt.Printf("ID: %d Type: %T Value: %s\n", a_id, allowedRepos, i)
if i == *repo.Name {
fmt.Printf("Index: %d, repo %s found in Allowed as %s\n", r_id, *repo.Name, i)
} else {
fmt.Printf("ALARM: repo %s was NOT found in Allowed!\n", *repo.Name)
}
}
}
...
Check:
$ go run go-github-public-repos-checker.go
Allowed repos: [org-repo-1-pub org-repo-2-pub]
ID: 0 Type: []string Value: org-repo-1-pub org-repo-2-pub
ALARM: repo org-repo-1-pub was NOT found in Allowed!
ID: 0 Type: []string Value: org-repo-1-pub org-repo-2-pub
ALARM: repo org-repo-2-pub was NOT found in Allowed!
allowedRepos is really a list but with the only 0 element which keeps the “org-repo-1-pub org-repo-2-pub” value.
Attempt Number Two
To make this work – we need to convert the allowedRepos to a real list.
Let’s use the strings package:
...
// allowedRepos := []string{os.Getenv("ALLOWED_REPOS")}
allowedRepos := strings.Fields(os.Getenv("ALLOWED_REPOS"))
fmt.Printf("Allowed repos: %s\n", allowedRepos)
for r_id, repo := range repos {
for a_id, i := range allowedRepos {
fmt.Printf("ID: %d Type: %T Value: %s\n", a_id, allowedRepos, i)
if i == *repo.Name {
fmt.Printf("Index: %d, repo %s found in Allowed as %s\n", r_id, *repo.Name, i)
} else {
fmt.Printf("ALARM: repo %s was NOT found in Allowed!\n", *repo.Name)
}
}
}
...
allowedRepos now will be filled with the strings.Fields().
Check result:
$ go run go-github-public-repos-checker.go
Allowed repos: [org-repo-1-pub org-repo-2-pub]
ID: 0 Type: []string Value: org-repo-1-pub
Index: 0, repo org-repo-1-pub found in Allowed as org-repo-1-pub
ID: 1 Type: []string Value: org-repo-2-pub
ALARM: repo org-repo-1-pub was NOT found in Allowed!
ID: 0 Type: []string Value: org-repo-1-pub
ALARM: repo org-repo-2-pub was NOT found in Allowed!
ID: 1 Type: []string Value: org-repo-2-pub
Index: 1, repo org-repo-2-pub found in Allowed as org-repo-2-pub
Okay – much better now but this also will not work as there are some “false-positive” results because each repos‘s element is compared with each allowedRepos‘s element.
Attempt Number Three
Let’s rewrite it and now let’s try to use a repo‘s indexes to choose an element from the allowedRepos:
...
for r_id, repo := range repos {
fmt.Printf("%d %s\n", r_id, *repo.Name)
if *repo.Name != allowedRepos[r_id] {
fmt.Printf("ALARM: repo %s was NOT found in Allowed!\n", *repo.Name)
} else {
fmt.Printf("Repo %s found in Allowed\n", *repo.Name)
}
}
...
I.e., from the repos, we getting an element’s ID and then checking the allowedRepos‘s element with the same ID.
Run:
$ go run go-github-public-repos-checker.go
Allowed repos: [org-repo-1-pub org-repo-2-pub]
0 org-repo-1-pub
Repo org-repo-1-pub found in Allowed
1 org-repo-2-pub
Repo org-repo-2-pub found in Allowed
Nice!
But another issue can happen now…
What if the order in both lists will differ?
Set instead of the:
$ export ALLOWED_REPOS="org-repo-1-pub org-repo-2-pub"
Repositories names in the reversed order:
$ export ALLOWED_REPOS="org-repo-2-pub org-repo-1-pub"
Check:
$ go run go-github-public-repos-checker.go
Allowed repos: [org-repo-2-pub org-repo-1-pub]
0 org-repo-1-pub
ALARM: repo org-repo-1-pub was NOT found in Allowed!
1 org-repo-2-pub
ALARM: repo org-repo-2-pub was NOT found in Allowed!
D’oh!
Attempt Number Four
The next idea was to use the reflect module and its DeepEqual() function.
Add one more list called actualRepos, add repositories taken from Github with the append and then compare them:
...
allowedRepos := strings.Fields(os.Getenv("ALLOWED_REPOS"))
var actualRepos []string
for _, repo := range repos {
actualRepos = append(actualRepos, *repo.Name)
}
fmt.Printf("Allowed: %s\n", allowedRepos)
fmt.Printf("Actual: %s\n", actualRepos)
fmt.Println("Slice equal: ", reflect.DeepEqual(allowedRepos, actualRepos))
...
Run:
$ go run go-github-public-repos-checker.go
Allowed: [org-repo-2-pub org-repo-1-pub]
Actual: [org-repo-1-pub org-repo-2-pub]
Slice equal: false
And again no… Although if revert order back – it will work:
$ export ALLOWED_REPOS="org-repo-1-pub org-repo-2-pub"
go run go-github-public-repos-checker.go
Allowed: [org-repo-1-pub org-repo-2-pub]
Actual: [org-repo-1-pub org-repo-2-pub]
Slice equal: true
So, we need to go back to the loop’s variant but to somehow check repo‘s value in the whole allowedRepos list and only if it’s absent – create an alert.
Attempt Number Five
The solution was next: create a dedicated function which will check all allowedRepos‘s elements in a loop and will return true if the value will be found and false otherwise. Then – we can use this in the main()‘s loop.
Let’s try.
Create a isAllowedRepo()function which will accept two arguments – a repository name to be checked and the allowed repositories list and will return a boolean value:
...
func isAllowedRepo(repoName string, allowedRepos []string) bool {
for _, i := range allowedRepos {
if i == repoName {
return true
}
}
return false
}
...
Then in the main() – run a loop over all repos‘s elements, pass them one by one to the isAllowedRepo() and then print a result:
...
for _, repo := range repos {
fmt.Printf("Checking %s\n", *repo.Name)
fmt.Println(isAllowedRepo(*repo.Name, allowedRepos))
}
...
Let’s test.
First, restore allowed repositories list in the initial order:
$ export ALLOWED_REPOS="org-repo-1-pub org-repo-2-pub"
go run go-github-public-repos-checker.go
Checking org-repo-1-pub
true
Checking org-repo-2-pub
true
Good!
And reversed order:
$ export ALLOWED_REPOS="org-repo-2-pub org-repo-1-pub"
go run go-github-public-repos-checker.go
Checking org-repo-1-pub
true
Checking org-repo-2-pub
true
Still works…
Now remove one of the allowed repositories:
$ export ALLOWED_REPOS="org-repo-2-pub"
go run go-github-public-repos-checker.go
Checking org-repo-1-pub
false
Checking org-repo-2-pub
true
Great!
Now when we will get false – we can raise an alert.
The whole code now. Leaving “as is” just for history:
package main
import (
"os"
"context"
"fmt"
_"reflect"
"strings"
"github.com/google/go-github/github"
)
func isAllowedRepo(repoName string, allowedRepos []string) bool {
for _, i := range allowedRepos {
if i == repoName {
return true
}
}
return false
}
func main() {
client := github.NewClient(nil)
opt := &github.RepositoryListByOrgOptions{Type: "public"}
repos, _, _ := client.Repositories.ListByOrg(context.Background(),
os.Getenv("GITHUB_ORG_NAME"), opt)
// allowedRepos := []string{os.Getenv("ALLOWED_REPOS")}
allowedRepos := strings.Fields(os.Getenv("ALLOWED_REPOS"))
// var actualRepos []string
/*
for _, repo := range repos {
actualRepos = append(actualRepos, *repo.Name)
}
fmt.Printf("Allowed: %s\n", allowedRepos)
fmt.Printf("Actual: %s\n", actualRepos)
fmt.Println("Slice equal: ", reflect.DeepEqual(allowedRepos, actualRepos))
*/
for _, repo := range repos {
fmt.Printf("Checking %s\n", *repo.Name)
fmt.Println(isAllowedRepo(*repo.Name, allowedRepos))
}
/*
for r_id, repo := range repos {
for _, i := range allowedRepos {
fmt.Printf("Checking %s and %s\n", *repo.Name, i)
if i == *repo.Name {
fmt.Printf("Index: %d, repo %s found in Allowed as %s\n", r_id, *repo.Name, i)
break
} else {
fmt.Printf("ALARM: repo %s was NOT found in Allowed!\n", *repo.Name)
}
}
}
*/
/*
for r_id, repo := range repos {
fmt.Printf("%d %s\n", r_id, *repo.Name)
if *repo.Name != allowedRepos[r_id] {
fmt.Printf("ALARM: repo %s was NOT found in Allowed!\n", *repo.Name)
} else {
fmt.Printf("Repo %s found in Allowed\n", *repo.Name)
}
}
*/
/*
for r_id, repo := range repos {
for a_id, i := range allowedRepos {
fmt.Printf("ID: %d Type: %T Value: %s\n", a_id, allowedRepos, i)
if i == *repo.Name {
fmt.Printf("Index: %d, repo %s found in Allowed as %s\n", r_id, *repo.Name, i)
} else {
fmt.Printf("ALARM: repo %s was NOT found in Allowed!\n", *repo.Name)
}
}
}
*/
}
Golang Slack
And the final thing is to add a Slack notification.
Use ashwanthkumar/slack-go-webhook here.
Configure Slack’s WebHook, get its URL.
Add a new sendSlackAlarm() function which will accept two arguments – repository name and its URL:
...
func sendSlackAlarm(repoName string, repoUrl string) {
webhookUrl := os.Getenv("SLACK_URL")
text := fmt.Sprintf(":scream: ALARM: repository *%s* was NOT found in Allowed!", repoName)
attachment := slack.Attachment{}
attachment.AddAction(slack.Action{Type: "button", Text: "RepoURL",
Url: repoUrl, Style: "danger"})
payload := slack.Payload{
Username: "Github checker",
Text: text,
Channel: os.Getenv("SLACK_CHANNEL"),
IconEmoji: ":scream:",
Attachments: []slack.Attachment{attachment},
}
err := slack.Send(webhookUrl, "", payload)
if len(err) > 0 {
fmt.Printf("error: %s\n", err)
}
}
...
Add the sendSlackAlarm() execution to the main() if the isAllowedRepo() returned false:
...
for _, repo := range repos {
fmt.Printf("\nChecking %s\n", *repo.Name)
if isAllowedRepo(*repo.Name, allowedRepos) {
fmt.Printf("OK: repo %s found in Allowed\n", *repo.Name)
} else {
fmt.Printf("ALARM: repo %s was NOT found in Allowed!\n", *repo.Name)
sendSlackAlarm(*repo.Name, *repo.HTMLURL)
}
}
...
repo.HTMLURL we found from the go doc github.Repository.
Add $SLACK_URL and $SLACK_CHANNEL environment variables:
$ export SLACK_URL="https://hooks.slack.com/services/T1641GRB9/BA***WRE"
$ export SLACK_CHANNEL="#general"
Restore the full repositories list on reversed order:
$ export ALLOWED_REPOS="org-repo-2-pub org-repo-1-pub"
Check:
$ go run go-github-public-repos-checker.go
Checking org-repo-1-pub
OK: repo org-repo-1-pub found in Allowed
Checking org-repo-2-pub
OK: repo org-repo-2-pub found in Allowed
Okay…
Remove one allowed:
$ export ALLOWED_REPOS="org-repo-2-pub"
Check again:
$ go run go-github-public-repos-checker.go
Checking org-repo-1-pub
ALARM: repo org-repo-1-pub was NOT found in Allowed!
Checking org-repo-2-pub
OK: repo org-repo-2-pub found in Allowed
And Slack notification:
Done!
The script is available in the setevoy-tools Github repository.
Similar Posts
History
- 15th April, 2019: Initial version
