Running interactive command line code from Jupyter notebook
Assuming you are asking about interactivity, there is something you can try.
If you ever wondered how Jupyter knows when the output of a cell ends: well, it apparently does not know, it just dumps any captured output into the most recently active cell:
import threading,time
a=5
threading.Thread(target=lambda:[print(a),time.sleep(20),print(a)]).start()
(deliberately shorter-than-nice example, as this is just side-info. While the 20-second wait is running you have time to activate another cell, perhaps via issuing an a=6
)
This can be used to get the output of some console code to the screen, while controlling it from the main thread:
import sys,threading,subprocess
proc=subprocess.Popen('/bin/sh',stdout=subprocess.PIPE,stdin=subprocess.PIPE,stderr=subprocess.STDOUT)
pout=proc.stdout
pin=proc.stdin
def outLoop():
running=True
while(running):
line=pout.readline().decode(sys.stdout.encoding)
print(line,end='')
running='\n' in line
print('Finished')
threading.Thread(target=outLoop).start()
Then you can isssue commands, like
pin.write(b'ls -l\n')
pin.flush()
and
pin.write(b'exit\n')
pin.flush()
Even b'ls\nexit\n'
works, that is why outLoop
is so long (a simple while(proc.poll() is None)
-print(...)
loop would finish sooner than it has grabbed all output.
Then the whole thing can be automated as:
while(proc.poll() is None):
inp=bytearray(input('something: ')+'\n',sys.stdin.encoding)
if(proc.poll() is None):
pin.write(inp)
pin.flush()
This works well on https://try.jupyter.org/, but obviously I did not want to try installing conda packages there, so I do not know what happens when conda asks a question.
A lucky thing is that the input field stays at the bottom of the cell (tested with ls;sleep 10;ls
). An unlucky thing is that the input field needs an extra entry at the end to disappear (and that is already the 'nice' way, when it was a simple while(...)
-write(bytearray(input()))
-flush()
3-liner, it was exiting with an exception.
If someone wants to try this on Windows, it works with 'cmd'
, but I suggest using a hardcoded 'windows-1252'
instead of sys.stdin/out.encoding
: they say UTF-8, but a simple dir
command already produces output which is neither UTF-8 nor ASCII (the non-breakable space between the 3-digit groups in sizes is a 0xA0 character). Or just remove the decode
part (and use running=0xA in line
)
the !command
syntax is an alternative syntax of the %system
magic, which documentation can be found here.
As you guessed, it's invoking os.system
and as far as os.system
works there is no simple way to know whether the process you will be running will need input from the user. Thus when using the notebook or any multi-process frontend you have no way to dynamically provide input to your the program you are running. (unlike a call to input
in Python we can intercept).
As you explicitly show interest in installing packages from the notebook, I suggest reading the following from Jake Van Der Plas which is a summary of a recent discussion on the subject, and explain some of the complications of doing so. You can of course go with --yes
option of conda, but it does not guarantee that installing with conda will always work.
Note also that !command
is an IPython feature, not a Jupyter one.