Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add aarch64 support, and more responsive qemu interaction #23

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 55 additions & 23 deletions ovmf-vars-generator
Original file line number Diff line number Diff line change
Expand Up @@ -20,19 +20,32 @@ import tempfile
import shutil
import string
import subprocess
import select


def strip_special(line):
return ''.join([c for c in str(line) if c in string.printable])


def generate_qemu_cmd(args, readonly, *extra_args):
if args.disable_smm:
is_x86 = True
arch_args = []
if os.path.basename(args.qemu_binary) == 'qemu-system-aarch64' or args.arch == 'aarch64':
Copy link

@dannf dannf Sep 22, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally I'd prefer we not make any assumptions about what the qemu_binary path means and just blindly use whatever they pass in. I think --arch should be the one and only way to determine what arguments are passed to that binary. If the user doesn't specify a binary, we can then pick a reasonable default based on the arch (where I think x86_64 would be the reasonable default).

machinetype = 'virt'
arch_args.extend(['-cpu', 'cortex-a57'])
is_x86 = False
elif args.disable_smm:
machinetype = 'pc'
else:
machinetype = 'q35,smm=on'
machinetype += ',accel=%s' % ('kvm' if args.enable_kvm else 'tcg')

if is_x86 == True:
arch_args.extend(['-chardev', 'pty,id=charserial1',
'-device', 'isa-serial,chardev=charserial1,id=serial1',
'-global', 'driver=cfi.pflash01,property=secure,value=%s' % (
'off' if args.disable_smm else 'on')])

if args.oem_string is None:
oemstrings = []
else:
Expand All @@ -50,18 +63,14 @@ def generate_qemu_cmd(args, readonly, *extra_args):
'-no-user-config',
'-nodefaults',
'-m', '768',
'-smp', '2,sockets=2,cores=1,threads=1',
'-chardev', 'pty,id=charserial1',
'-device', 'isa-serial,chardev=charserial1,id=serial1',
'-global', 'driver=cfi.pflash01,property=secure,value=%s' % (
'off' if args.disable_smm else 'on'),
'-smp', '1,sockets=1,cores=1,threads=1',
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why have less CPUs on ARM?

Copy link
Author

@jlinton jlinton Sep 23, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't remember exactly why I reduced that, but IIRC it was because initially I was testing this in a VM with only a single core.

Frankly, a better question is why have more than 1 core, the entire firmware stack is single threaded.
A better change is probably to completely remove that line as its unnecessary.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Frankly, a better question is why have more than 1 core, the entire firmware stack is single threaded.

OVMF includes modules for providing EFI_PEI_MP_SERVICES_PPI, EFI_MP_SERVICES_PROTOCOL, and EFI_MM_MP_PROTOCOL. So the general statement that the entire firmware stack is single threaded is incorrect.

It is true that PI / UEFI services are generally only usable on the BSP.

It is also true that, for this particular use case, we do not need more than 1 VCPU.

'-drive',
'file=%s,if=pflash,format=raw,unit=0,readonly=on' % (
args.ovmf_binary),
'-drive',
'file=%s,if=pflash,format=raw,unit=1,readonly=%s' % (
args.out_temp, 'on' if readonly else 'off'),
'-serial', 'stdio'] + oemstrings + list(extra_args)
'-serial', 'stdio'] + oemstrings + arch_args + list(extra_args)


def download(url, target, suffix, no_download):
Expand Down Expand Up @@ -95,17 +104,17 @@ def enroll_keys(args):
args,
False,
'-drive',
'file=%s,format=raw,if=none,media=cdrom,id=drive-cd1,'
'readonly=on' % args.uefi_shell_iso,
'-device',
'ide-cd,drive=drive-cd1,id=cd1,'
'bootindex=1')
'file=%s,format=raw,if=virtio,media=cdrom,id=drive-cd1,'
'readonly=on' % args.uefi_shell_iso)
p = subprocess.Popen(cmd,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT)
logging.info('Performing enrollment')
# Wait until the UEFI shell starts (first line is printed)
pollobj = select.poll()
pollobj.register(p.stdout, select.POLLIN)

read = p.stdout.readline()
if b'char device redirected' in read:
read = p.stdout.readline()
Expand All @@ -116,25 +125,46 @@ def enroll_keys(args):
if args.print_output:
print(strip_special(read), end='')
print()
# Send the escape char to enter the UEFI shell early
p.stdin.write(b'\x1b')
p.stdin.flush()
# And then run the following three commands from the UEFI shell:
# change into the first file system device; install the default
# keys and certificates, and reboot
p.stdin.write(b'fs0:\r\n')
p.stdin.write(b'EnrollDefaultKeys.efi\r\n')
p.stdin.write(b'reset -s\r\n')
# Send the press enter for the default boot (uefi shell)
p.stdin.write(b'\r\n')
p.stdin.flush()

wait_timeout = 100;
while True:
read = p.stdout.readline()
poll_result = pollobj.poll(1000)
if poll_result:
# readline can get stuck in menus and other
# UI elements in uefi which don't respond
# with a CR the poll above doesn't help
# with that case.
read = p.stdout.readline()
else:
wait_timeout-=1
if wait_timeout == 0:
logging.info('Failed enrollment')
# consider something stronger here
# as we can still get stuck in the p.wait()
break;
continue
if args.print_output and len(read):
print('OUT: %s' % strip_special(read), end='')
print()
if b'info: success' in read:
if b'seconds to skip' in read:
p.stdin.write(b'\r\n')
p.stdin.flush()
elif b'info: success' in read:
break
elif b'Reset with <null string>' in read:
break
elif b'Shell>' in read:
# And then run the following three commands from the UEFI shell:
# change into the first file system device; install the default
# keys and certificates, and reboot
p.stdin.write(b'fs0:\r\n')
p.stdin.write(b'EnrollDefaultKeys.efi\r\n')
p.stdin.write(b'reset -s\r\n')
p.stdin.flush()

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which serves a purpose if one is in a python env. OTOH, python isn't a great shell scripting language anymore than c. Worse, this script is being used on fedora in an actual rpmbuild/bash environment. In that case about 80% of this scripts functionality can be represented like:

/usr/bin/expect <<EOF
spawn /usr/bin/qemu-system-${MACHTYPE} ${QEMU_ARCH} ${COMMON} $FIRMWARE ${VARS}=off $CDROM $SMBIOS
expect -exact "Shell>"
send -- "\r\nfs0:\r\n"
send -- "EnrollDefaultKeys.efi\r\n"
send -- "reset -s\r\n"
expect eof
EOF

the remainder, which I've also done (dealing with multiple arches, and validating enrolled keys) is about 3-4x the size of that.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which serves a purpose if one is in a python env. OTOH, python isn't a great shell scripting language anymore than c. Worse, this script is being used on fedora in an actual rpmbuild/bash environment.

I'm not arguing for/against this being in python vs. sh, just that since it is in python, pexpect is at our disposal.

p.wait()
if args.print_output:
print(strip_special(p.stdout.read()), end='')
Expand Down Expand Up @@ -182,6 +212,8 @@ def test_keys(args):

def parse_args():
parser = argparse.ArgumentParser()
parser.add_argument('--arch', '-a', help='Architecture hint',
choices=['x86_64', 'aarch64'])
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest using the UEFI names x64 & aarch64, since that's really what we're referring to here.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add default='x86_64' (or 'x64') here?

parser.add_argument('output', help='Filename for output vars file')
parser.add_argument('--out-temp', help=argparse.SUPPRESS)
parser.add_argument('--force', help='Overwrite existing output file',
Expand Down