December 1, 2010

Python - Search a Local or Remote Splunk Server

Some basic instructions for searching Splunk from Python...

First, you must install Splunk on the machine you will run the Python script from. Splunk installs its own Python interpreter that you can use to run your code. I am using Splunk 4.14, which includes Python 2.6.

(It looks like you can set some environment variables and install a few Python dependencies along with the Python SDK and get this going "outside" of Splunk. But the easiest option is just to run on their interpreter).

To run your own Python scripts on Splunk's interpreter:
- save script into Splunk's "bin" directory
(usually "/opt/splunk/bin" or "C:\Program Files\Splunk\bin")
- go to the "bin" directory and run:
splunk cmd python your_script.py

So...
What goes in your Python code?

First, import the modules you will need:

 
import time
import splunk.auth
import splunk.search

Next, authenticate and get a session key.

For the local splunk host:

 
key = splunk.auth.getSessionKey('user', 'password')

If you are going to search a remote splunk host, you must authenticate against it by adding the "hostPath" parameter:

 
key = splunk.auth.getSessionKey('user', 'password', hostPath='https://mysplunk:8089')

Tips:
- use https, even if you are not using ssl in your splunk web interface
- 'admin' user doesn't seem to work. user a normal user/password.

Next, submit a search job.

For a local search:

 
job = splunk.search.dispatch('search index="os" *', earliest_time='-15m')

For a remote search, use the "hostPath" parameter again:

 
job = splunk.search.dispatch('search index="os" *', earliest_time='-15m', hostPath='https://mysplunk:8089')

print the job details:

 
print job

wait for the results:

 
while not job.isDone:
    time.sleep(.25)    

print results

 
for result in job.results:
    print result

Altogether in a Python script:

 
#!/usr/bin/env python
# Corey Goldberg - 2010
# 
#  search a remote splunk server
#
#  instructions:
#   - save script into splunk's "bin" directory
#     (usually "/opt/splunk/bin" or "C:\Program Files\Splunk\bin")
#   - go to the "bin" directory and run: 
#     $ splunk cmd python my_script.py
#



import time
import splunk.auth
import splunk.search



SPLUNK_SERVER = '192.168.12.173'
USER_NAME = 'foo'
PASSWORD = 'secret'
SEARCH_STRING = 'search index="os"'
EARLIEST_TIME = '-15m'



def main():
    # authenticate
    key = splunk.auth.getSessionKey(USER_NAME, PASSWORD, hostPath='https://%s:8089' % SPLUNK_SERVER)
    print 'auth key:\n%s' % key
    
    # submit a search job
    job = splunk.search.dispatch(SEARCH_STRING, earliest_time=EARLIEST_TIME, hostPath='https://%s:8089' % SPLUNK_SERVER)
    print 'job details:\n%s' % job

    # wait for results
    while not job.isDone:
        time.sleep(.25)
    
    print 'results:'    
    for result in job.results:
        print result
          
          

if __name__== '__main__':
    main()

1 comment:

ampledata said...

Hi Corey,
Per your comment "'admin' user doesn't seem to work." Please see $SPLUNK_HOME/etc/system/README/server.conf.spec:


allowRemoteLogin =
* Controls remote management by restricting general login.
* If 'always', all remote logins are allowed.
* If 'never', only local logins to splunkd will be allowed. Note that this will still allow
remote management through splunkweb if splunkweb is on the same server.
* If 'requireSetPassword' (default):
* In the free license, remote login is disabled.
* In the pro license, remote login is only disabled for the admin user that has not changed their default password.


Your two options are to change the default admin password, or add this line to $SPLUNK_HOME/etc/system/local/server.conf:
allowRemoteLogin = always

Thanks,
gba@splunk.com