Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proxy support not working if using the environment #16

Closed
ph opened this issue Mar 23, 2015 · 7 comments
Closed

Proxy support not working if using the environment #16

ph opened this issue Mar 23, 2015 · 7 comments

Comments

@ph
Copy link

ph commented Mar 23, 2015

I am currently looking to add the proxy support back into logstash 1.5, but I am unable to make the jars download uses a proxy. I was expecting setting the http_proxy env variable to an appropriate proxy and the libs management would pick it up.

Versions:

  • jar-dependencies 0.1.7
  • ruby-maven 3.1.1.0.8

Scenarios

  1. I've looked a bit at the codebase of both rubygems and jar-dependencies, the current implementation of jar-dependencies uses the Gem.configuration[:proxy] but in the rubygems test suite they uses Gem.configuration[:http_proxy]. After making my code update both the http_proxy and the proxy key and adding trace I see that my configuration is correctly send to the mvn class loader (in ruby maven)
    but the jar download isn't using the proxy.
  2. If I create a a simple settings.xml and set the JARS_MAVEN_SETTINGS to use it, it doesn't go to the proxy..
  3. If I move the same settings.xml file in the ~/.m2 directory, the jars download will go through the proxy

Settings.xml

<settings>
  <proxies>
   <proxy>
      <id>example-proxy</id>
      <active>true</active>
      <protocol>http</protocol>
      <host>127.0.0.1</host>
      <port>3128</port>
    </proxy>
  </proxies>
</settings>

Logstash ref: elastic/logstash#2851

@ph
Copy link
Author

ph commented Mar 23, 2015

@mkristian
Copy link
Owner

fixed the maven-settings issue.

the proxy support from Gem.configuration is still pending - not sure yet how to fix it

@ph
Copy link
Author

ph commented Oct 14, 2016

@mkristian This is still an issue today (jar dependencies 0.3.5), I've run into it and I have a setup to reproduce it. Do you want to reopen it or create a new one?

I am using a 2 vagrant VM, one with squid3 and the other vm only has access to the squid proxy.
When I create the settings.xml it fixes the problem.

#/etc/squid3/squid.conf
#Recommended minimum configuration:
acl localhost src 127.0.0.1/32
acl to_localhost dst 127.0.0.0/8
acl localnet src 0.0.0.0/8 192.168.100.0/24 192.168.1.0/24
acl SSL_ports port 443
acl Safe_ports port 80      # http
acl Safe_ports port 21      # ftp
acl Safe_ports port 443     # https
acl Safe_ports port 70      # gopher
acl Safe_ports port 210     # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280     # http-mgmt
acl Safe_ports port 488     # gss-http
acl Safe_ports port 591     # filemaker
acl Safe_ports port 777     # multiling http

acl CONNECT method CONNECT

http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports

http_access deny to_localhost
icp_access deny all
htcp_access deny all

http_port 3128
hierarchy_stoplist cgi-bin ?
access_log /var/log/squid3/access.log squid


#Suggested default:
refresh_pattern ^ftp:       1440    20% 10080
refresh_pattern ^gopher:    1440    0%  1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern .       0   20% 4320
# Leave coredumps in the first cache dir

@mkristian mkristian reopened this Oct 15, 2016
@mkristian
Copy link
Owner

@ph will look into this asap

mkristian added a commit that referenced this issue Nov 7, 2016
@suyograo
Copy link

@mkristian any ETA on when we can expect a new version with #16? Thanks much for the fix.

@mkristian
Copy link
Owner

@suyograo getting the tests green on the weekend and then I release it and update jruby to bundle the new release

@suyograo
Copy link

@mkristian we included 0.3.7 in Logstash 5.1.1, and a user is reporting what looks like a bug in jar_dependencies. See elastic/logstash#6396 for details.

trombik pushed a commit to reallyenglish/ansible-role-logstash that referenced this issue Dec 13, 2016
I cannot test this locally because:

* logstash 5.x is not in the official package tree
* the local package can be retrieved over VPN
* but my local VPN client keeps crashing after a while
* SSH proxy works but a Java package `logstash` depends on does not
  respect HTTP_PROXY

"HTTP_PROXY and HTTPS_PROXY environment variables are not correctly
respected when jar-dependency fetch the pom"
elastic/logstash#6044 (comment)

"Proxy support not working if using the environment"
mkristian/jar-dependencies#16
mkristian/jar-dependencies#16 (comment)
trombik pushed a commit to reallyenglish/ansible-role-logstash that referenced this issue Dec 17, 2016
* s/plugin/logstash-plugin/g

`plugin` is deprecated. recent logstash documentation refer to
`logstash-plugin`.

while here remove `when: ansible_os_family == 'FreeBSD'`.

* logstash-plugin is also installed without exec bit

* support ubuntu

* introduce logstash_package_name

because FreeBSD has multiple version of logstash

* try logstash 5.x

I cannot test this locally because:

* logstash 5.x is not in the official package tree
* the local package can be retrieved over VPN
* but my local VPN client keeps crashing after a while
* SSH proxy works but a Java package `logstash` depends on does not
  respect HTTP_PROXY

"HTTP_PROXY and HTTPS_PROXY environment variables are not correctly
respected when jar-dependency fetch the pom"
elastic/logstash#6044 (comment)

"Proxy support not working if using the environment"
mkristian/jar-dependencies#16
mkristian/jar-dependencies#16 (comment)

* enable DEBUG

* more debug flags

* mount fdescfs and procfs

also, remove DEBUG as it turns out that sysutils/logstash5 has a bug
that removes exec bit of bundled gems' binaries. the port has been
fixed.

* the port has been merged

* use canonical port name

* s/fdesc/procfs/

* support logstash5

* wait for logstash after starting it

in the first ansible play, it restarts logstash immediately after
starting. this probably causes a race.

* the default port is 9600

* validate logstash configuration files

as there are multiple files, you cannot use `validate:`.

* a typo

* reload logstash instead of restarting it

when the change is something in configuration files.

* swap single quote and double quote

* remove TODOs

* pipeline.unsafe_shutdown can be enabled in logstash.yml
* visualvm should be handled by its own role
* config.reload.automatic can be enabled in logstash.yml
* reload handler has been implemented
* with ansible-role-redis, you can implement queuing
* plugins has been managed by the role

* comment why pif file is not used

* always run logstash as user logstash

as logstash creates directory and files when invoked.

if logstash binaries, including logstash-plugin, runs as root, it causes
subsequent logstash run to fail.

* fix wrong keys

* use logstash_config_merged instead of logstash_config

* wait for logstash to start in handler

there is a race when HUP terminates restart

* fix a typo

* document the finding

* manage jvm.options

* add other defaults

* document new valiables

* add License

* add some comments

* add comments on the task and the template

with new configuration option system, the task and the template will be
removed

* update the example

* add some notices

* restore logstash_config

no, rc.conf will not be removed as 'conf.d' must bedefined.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants