Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

added region flag to avoid having to parse the endpoint to get the region #1

Merged
merged 1 commit into from
Oct 16, 2020
Merged

added region flag to avoid having to parse the endpoint to get the region #1

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 26 additions & 8 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,8 @@ brew install aws-es-proxy
### Build from Source

#### Dependencies:
* go1.14+

- go1.14+

```sh
#requires go1.14
Expand Down Expand Up @@ -82,8 +83,6 @@ export AWS_SECRET_ACCESS_KEY=MY-SECRET-KEY
}
```



## Usage example:

You can use either argument `-endpoint` OR environment variable `ENDPOINT` to specify AWS ElasticSearch endpoint.
Expand All @@ -100,14 +99,14 @@ export ENDPOINT=https://test-es-somerandomvalue.eu-west-1.es.amazonaws.com
Listening on 10.0.0.1:9200
```

*aws-es-proxy* listens on 127.0.0.1:9200 if no additional argument is provided. You can change the IP and Port passing the argument `-listen`
_aws-es-proxy_ listens on 127.0.0.1:9200 if no additional argument is provided. You can change the IP and Port passing the argument `-listen`

```sh
./aws-es-proxy -listen :8080 -endpoint ...
./aws-es-proxy -listen 10.0.0.1:9200 -endpoint ...
```

By default, *aws-es-proxy* will not display any message in the console. However, it has the ability to print requests being sent to Amazon Elasticsearch, and the duration it takes to receive the request back. This can be enabled using the option `-verbose`
By default, _aws-es-proxy_ will not display any message in the console. However, it has the ability to print requests being sent to Amazon Elasticsearch, and the duration it takes to receive the request back. This can be enabled using the option `-verbose`

```sh
./aws-es-proxy -verbose ...
Expand All @@ -124,6 +123,10 @@ For a full list of available options, use `-h`:
```sh
./aws-es-proxy -h
Usage of ./aws-es-proxy:
-auth
Require HTTP Basic Auth
-debug
Print debug messages
-endpoint string
Amazon ElasticSearch Endpoint (e.g: https://dummy-host.eu-west-1.es.amazonaws.com)
-listen string
Expand All @@ -132,15 +135,30 @@ Usage of ./aws-es-proxy:
Log user requests and ElasticSearch responses to files
-no-sign-reqs
Disable AWS Signature v4
-password string
HTTP Basic Auth Password
-pretty
Prettify verbose and file output
-realm string
Authentication Required
-remote-terminate
Allow HTTP remote termination
-timeout int
Set a request timeout to ES. Specify in seconds, defaults to 15 (default 15)
-username string
HTTP Basic Auth Username
-verbose
Print user requests
-version
Print aws-es-proxy version
-region
AWS region (ex. us-west-2) (Required)
-insecure
Will not verify SSL (default false)
```


## Using HTTP Clients

After you run *aws-es-proxy*, you can now open your Web browser on [http://localhost:9200](http://localhost:9200). Everything should be working as you have your own instance of ElasticSearch running on port 9200.
After you run _aws-es-proxy_, you can now open your Web browser on [http://localhost:9200](http://localhost:9200). Everything should be working as you have your own instance of ElasticSearch running on port 9200.

To access Kibana, use [http://localhost:9200/_plugin/kibana/app/kibana](http://localhost:9200/_plugin/kibana/app/kibana)
To access Kibana, use [http://localhost:9200/\_plugin/kibana/app/kibana](http://localhost:9200/_plugin/kibana/app/kibana)
144 changes: 74 additions & 70 deletions aws-es-proxy.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ package main
import (
"bytes"
"crypto/subtle"
"crypto/tls"
"encoding/json"
"flag"
"fmt"
Expand All @@ -22,7 +23,6 @@ import (
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/credentials"
"github.com/aws/aws-sdk-go/aws/credentials/stscreds"
"github.com/aws/aws-sdk-go/aws/endpoints"
"github.com/aws/aws-sdk-go/aws/session"
v4 "github.com/aws/aws-sdk-go/aws/signer/v4"
"github.com/sirupsen/logrus"
Expand Down Expand Up @@ -69,23 +69,25 @@ type responseStruct struct {
}

type proxy struct {
scheme string
host string
region string
service string
endpoint string
verbose bool
prettify bool
logtofile bool
nosignreq bool
fileRequest *os.File
fileResponse *os.File
credentials *credentials.Credentials
httpClient *http.Client
auth bool
username string
password string
realm string
scheme string
host string
region string
service string
endpoint string
verbose bool
prettify bool
logtofile bool
nosignreq bool
fileRequest *os.File
fileResponse *os.File
credentials *credentials.Credentials
httpClient *http.Client
auth bool
username string
password string
realm string
remoteTerminate bool
insecure bool
}

func newProxy(args ...interface{}) *proxy {
Expand All @@ -99,25 +101,33 @@ func newProxy(args ...interface{}) *proxy {
CheckRedirect: noRedirect,
}

if args[12].(bool) == true {
client.Transport = &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
}
}

return &proxy{
endpoint: args[0].(string),
verbose: args[1].(bool),
prettify: args[2].(bool),
logtofile: args[3].(bool),
nosignreq: args[4].(bool),
httpClient: &client,
auth: args[6].(bool),
username: args[7].(string),
password: args[8].(string),
realm: args[9].(string),
endpoint: args[0].(string),
verbose: args[1].(bool),
prettify: args[2].(bool),
logtofile: args[3].(bool),
nosignreq: args[4].(bool),
httpClient: &client,
auth: args[6].(bool),
username: args[7].(string),
password: args[8].(string),
realm: args[9].(string),
remoteTerminate: args[10].(bool),
region: args[11].(string),
insecure: args[12].(bool),
}
}

func (p *proxy) parseEndpoint() error {
var (
link *url.URL
err error
isAWSEndpoint bool
link *url.URL
err error
)

if link, err = url.Parse(p.endpoint); err != nil {
Expand Down Expand Up @@ -146,6 +156,9 @@ func (p *proxy) parseEndpoint() error {
p.scheme = link.Scheme
p.host = link.Host

p.service = "es"
logrus.Debugln("AWS Region", p.region)

// AWS SignV4 enabled, extract required parts for signing process
if !p.nosignreq {

Expand All @@ -154,29 +167,6 @@ func (p *proxy) parseEndpoint() error {
if len(split) < 2 {
logrus.Debugln("Endpoint split is less than 2")
}

awsEndpoints := []string{}
for _, partition := range endpoints.DefaultPartitions() {
for region := range partition.Regions() {
awsEndpoints = append(awsEndpoints, fmt.Sprintf("%s.es.%s", region, partition.DNSSuffix()))
}
}

isAWSEndpoint = false
for _, v := range awsEndpoints {
if split[1] == v {
logrus.Debugln("Provided endpoint is a valid AWS Elasticsearch endpoint")
isAWSEndpoint = true
break
}
}

if isAWSEndpoint {
// Extract region and service from link. This should be save now
parts := strings.Split(link.Host, ".")
p.region, p.service = parts[1], "es"
logrus.Debugln("AWS Region", p.region)
}
}

return nil
Expand Down Expand Up @@ -210,6 +200,10 @@ func (p *proxy) getSigner() *v4.Signer {
}

func (p *proxy) ServeHTTP(w http.ResponseWriter, r *http.Request) {
if p.remoteTerminate && r.URL.Path == "/terminate-proxy" && r.Method == http.MethodPost {
logrus.Infoln("Terminate Signal")
os.Exit(0)
}

if p.auth {
user, pass, ok := r.BasicAuth()
Expand Down Expand Up @@ -330,6 +324,7 @@ func (p *proxy) ServeHTTP(w http.ResponseWriter, r *http.Request) {

fmt.Println()
fmt.Println("========================")
fmt.Println("Region: ", p.region)
fmt.Println(t.Format("2006/01/02 15:04:05"))
fmt.Println("Remote Address: ", r.RemoteAddr)
fmt.Println("Request URI: ", proxied.RequestURI())
Expand Down Expand Up @@ -420,22 +415,25 @@ func copyHeaders(dst, src http.Header) {
func main() {

var (
debug bool
auth bool
username string
password string
realm string
verbose bool
prettify bool
logtofile bool
nosignreq bool
ver bool
endpoint string
listenAddress string
fileRequest *os.File
fileResponse *os.File
err error
timeout int
debug bool
auth bool
username string
password string
realm string
verbose bool
prettify bool
logtofile bool
nosignreq bool
ver bool
endpoint string
listenAddress string
fileRequest *os.File
fileResponse *os.File
err error
timeout int
remoteTerminate bool
region string
insecure bool
)

flag.StringVar(&endpoint, "endpoint", "", "Amazon ElasticSearch Endpoint (e.g: https://dummy-host.eu-west-1.es.amazonaws.com)")
Expand All @@ -451,6 +449,9 @@ func main() {
flag.StringVar(&username, "username", "", "HTTP Basic Auth Username")
flag.StringVar(&password, "password", "", "HTTP Basic Auth Password")
flag.StringVar(&realm, "realm", "", "Authentication Required")
flag.BoolVar(&remoteTerminate, "remote-terminate", false, "Allow HTTP remote termination")
flag.StringVar(&region, "region", "", "AWS Region (ex. us-west-2)")
flag.BoolVar(&insecure, "insecure", false, "Verify SSL")
flag.Parse()

if endpoint == "" {
Expand Down Expand Up @@ -496,6 +497,9 @@ func main() {
username,
password,
realm,
remoteTerminate,
region,
insecure,
)

if err = p.parseEndpoint(); err != nil {
Expand Down