Skip to content
This repository has been archived by the owner on Oct 11, 2023. It is now read-only.

Developer Reference Guide

Jump to bottom
Hector Garcia Tellado edited this page Oct 31, 2017 · 1 revision

This article provides a list of frequently used tasks in the day-to-day development process in the Remote monitoring solution.

For troubleshooting help to identify and resolve commonly encountered issues, please visit our Developer Troubleshooting Guide

Contents

Managing a Solution

Managing Deployed Containers

Deployment

Local Development

Customization

For other customization options, please read the Customize the solution tutorial on docs.microsoft.com

Managing a Solution

Add a user to a solution

In order to allow accounts that are not part of your subscription to have access, you must give them access through the Azure portal.

  1. TODO

Connect a physical device

For detailed instructions on how to connect a physical devices, please read the Connect physical device tutorial on docs.microsoft.com

Managing Deployed Containers

SSH into a VM

Prerequisites

  • To SSH into a VM, need to know your VM sign in information from deployment
  • Find your VM's SSH address from the Azure Portal.
    • Navigate to the VM in the Azure Portal (portal.azure.com > Resource Groups > Select Resource Group for your PCS > Select Virtual Machine)
    • You can also select "Connect" from your VM's portal page which will display your ssh command.

Sign into a VM

  • To sign into a VM using bash, use your username followed by the IP address of the VM and enter your sign in credentials. 
    ssh myusername@12.34.567.890

Disable authentication

In order to test HTTP requests from tools like Postman you will need to disable authentication in the application.

  1. Open a console and SSH into your Azure VM (ssh {user}@{host}) using the password chosen during deployment.

    • You can also find the VM connection information in the Azure Portal at
      portal.azure.com > Resource Groups > Select Resource Group for your PCS > Select Virtual Machine > Connect

  2. run sudo nano /app/webui-config.js to edit the webui config file

  3. change authEnabled: true, to authEnabled: false,

  4. press CTRL+X, then press Y to save and exit from the editor

  5. run sudo nano /app/start.sh to edit the startup script and add export PCS_AUTH_REQUIRED="false" just after the first line.

  6. press CTRL+X, then press Y to save and exit from the editor

  7. restart the service by running sudo ./app/start.sh

Common Docker Commands

From the VM you can accomplish many common tasks with the following commands. For more information see the Docker command line documentation

Pre-requisites

List all images that are locally stored with the Docker engine

docker images

List all running containers to check status of a service

docker ps -a

Example output:

CONTAINER ID        IMAGE                                            COMMAND                  CREATED             STATUS              PORTS                                      NAMES
b84eeb26c07a        azureiotpcs/remote-monitoring-nginx:latest       "/bin/bash /app/ru..."   12 minutes ago      Up 12 minutes       0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp   app_reverseproxy_1
c66af1abbafb        azureiotpcs/pcs-remote-monitoring-webui:latest   "/bin/sh /app/run.sh"    12 minutes ago      Up 12 minutes       80/tcp, 443/tcp                            app_webui_1
8cb29cd3902d        azureiotpcs/telemetry-agent-dotnet:latest        "/bin/bash /app/ru..."   13 minutes ago      Up 13 minutes                                                  app_telemetryagent_1
d96b506acf7a        azureiotpcs/pcs-config-dotnet:latest             "/bin/bash /app/ru..."   13 minutes ago      Up 13 minutes                                                  app_config_1
6c6802aacd91        azureiotpcs/device-simulation-dotnet:latest      "/bin/bash /app/ru..."   13 minutes ago      Up 13 minutes                                                  app_devicesimulation_1
47ec9f594920        azureiotpcs/telemetry-dotnet:latest              "/bin/bash /app/ru..."   13 minutes ago      Up 13 minutes                                                  app_telemetry_1
3531b19916be        azureiotpcs/pcs-storage-adapter-dotnet:latest    "/bin/bash /app/ru..."   13 minutes ago      Up 13 minutes                                                  app_storageadapter_1
d84d888d55e3        azureiotpcs/pcs-auth-dotnet                      "/bin/bash /app/ru..."   13 minutes ago      Up 13 minutes                                                  app_auth_1
67dbea599512        azureiotpcs/iothub-manager-dotnet                "/bin/bash /app/ru..."   14 minutes ago      Up 14 minutes                                                  app_iothubmanager_1

Get the logs for a running container

docker logs {Container ID}
docker logs --tail 100 {Container ID} (lists last 100 lines of log file)

Pull the latest image from Docker Hub

docker pull {docker-hub-user}/{docker-hub-container}:latest
Ex: docker pull azureiotpcs/iothub-manager-dotnet:latest

Stop a running container

docker stop {container-id}

Start simulations from VM

  1. Sign into the vm with the instructions above

  2. navigate to the app folder from the root directory: cd /app/

  3. run the simulation script from the terminal (root access required)

    sudo ./simulate.sh

    This will create the default simulations with the following command

    POST https://localhost/devicesimulation/v1/simulations?template=default

Pull the logs to a local machine for a container

  1. Open a bash terminal.
  2. SSH into the VM using the instructions above in "SSH into a VM".
  3. Find your container name or id with docker ps.
  4. Write the logs for a container to disk: docker logs <containerid> >& simlog.log
  5. Ensure the log data is present: sudo nano container.log
  6. Open a second bash terminal on your local machine.
  7. Copy the file local: scp <username>@<vm ip>:container.log c:/temp/container.log

Deployment

Deploy a PCS from the command line

  1. Clone the pcs-cli repo 
    git clone https://github.com/Azure/pcs-cli.git
  2. Follow the instructions in the README.md for How to use the CLI

Replace a running container with the testing images from PCS Docker Hub

These instructions are for pulling the testing images (the images in development) from the PCS Docker Hub repository from https://hub.docker.com/r/azureiotpcs/, to add your own images, please see Deploying containers from custom docker-hub account

Basic Deployments

  1. SSH into your deployment's VM using the instructions above
  2. navigate to the app folder from the root directory: cd /app
  3. Change to the /app directory: 
    cd /app
  4. Edit the docker-compose.yml file: 
    sudo nano docker-compose.yml
  5. Change your containers to use the :testing tag, e.g. 
    services:
  reverseproxy:
    image: azureiotpcs/remote-monitoring-nginx:testing
...
  webui:
    image: azureiotpcs/pcs-remote-monitoring-webui:testing
...
  auth:
    image: azureiotpcs/pcs-auth-dotnet:testing
  6. Save the file: Ctrl-X, Ctrl-Y
  7. Execute the start.sh command. sudo ./start.sh

To update all containers and environment variables

To reset all of the environment variables needed for your containers and redeploy the services based on the images specified in dockerhub (defined in your docker-compose.yml file).

run the start.sh script from the command line (root access required) sudo ./start.sh

Standard

Prerequisites

  1. Clone the pcs-cli repo
  2. Install az
  3. Install kubectl
  4. Access to the Subscription that created the deployment

One-time setup to create a container service for Kubernetes

  1. az login
  2. az account set --subscriptions {your-azure-subscription-id}
  3. Output .ssh credentials for Kubernetes
    az acs create -n {myClusterName} -d {myDNSPrefix} -g {resouceGroup} -t kubernetes --generate-ssh-keys
  4. Set credentials for Kubernetes
    az acs kubernetes get-credentials -g {myResorceGroupName} -n {myClusterName} --ssh-key-file ~\.ssh\{path-to-ssh-key}

Deploy Docker images through Kubernetes

Verify that you have access with kubectl get nodes

  1. kubectl create -f .\remotemonitoring\scripts\nginx-ingress-controller.yaml
  2. Go to your resource group on portal.azure.com and set up friendly DNS name for Public IP address that got created in (One-time setup to create a container service for Kubernetes)[#one-time-setup-to-create-a-container-service-for-kubernetes]. It will start with {myClusterName}. To confirm, match the IP address with "LoadBalancer Ingress" by running kubectl describe svc nginx-ingress
  3. Add actual values in the ConfigMap section in file all-in-one.yaml and deployment-configmap.yaml. Values to replace will be of format "{...}". Some examples below.
    • {DNS} with value from step 2
    • {IoT Hub connection string}
    • {DocumentDB connection string}
  4. kubectl create -f .\remotemonitoring\scripts\all-in-one.yaml

Important

If your account doesn't have the Azure Active Directory (AAD) and subscription permissions to create a service principal, then the command generates an error similar to Insufficient privileges to complete the operation.
Also, when using --generate-ssh-keys, if one key already exists at ~/.ssh/id_rsa then it will be used.

View the Kubernetes Dashboard for Standard Deployments

To view Kubernetes dashboard, run the following command, which will start a local web proxy for your cluster (it will start a local server at http://127.0.0.1:8001/ui):

az acs kubernetes browse -g {myResourceGroupName} -n {myClusterName} --ssh-key-file {path to ssh file}

Deploying containers from custom docker-hub account

In order to deploy your own changes to a pcs deployment, you will need to edit the build and deployment scripts in {repo}/scripts/docker to upload the containers to a docker-hub account

Prerequisites

  1. Clone the pcs-cli repo
  2. Create a Docker Hub account
  3. Create a Docker Hub repo for each microservice that you would like to redepoly. For example, if you would like to make changes to the Iot Hub Manager .NET project, you would create a docker hub repo for {your-username}/iothub-manager-dotnet

Instructions

Update project with custom changes

  1. Make changes to whatever microservices you would like
  2. When you are satisfied with your results, increment the version number on your project
    • Version file located in...
      • {repo}/version for .NET projects
      • build.sbt file for java projects

Modify the docker scripts with your docker hub repo info

In every repository there is a /scripts/docker folder with scripts to build, run, and publish Docker images. For the repositories that you have made changes to, update this script with your own Docker Hub repository info:

  • Update the following lines with your own information in the files listed below:
    DOCKER_IMAGE="{your-docker-hub-username/{your-docker-hub-repo}"

    • build
    • build.cmd
    • publish
    • publish.cmd
    • run
    • run.cmd

    image: {your-docker-hub-username/{your-docker-hub-repo}

    • docker-compose.yml

Generate new Docker images and push to Docker Hub

For each project that you have made changes to you will need to publish new Docker images to Docker Hub using your docker-hub account.

  1. cd {your-repo}/scripts/docker

  2. Build Docker image \

    • Windows cmd prompt:build
    • bash: ./build

  3. .NET repos only - Tag with "latest"
    docker tag {image-id-from-build} {your-docker-hub-username/{your-docker-hub-repo}:latest

  4. Publish docker image:

    • Windows cmd prompt:publish
    • bash: ./publish

  5. In the pcs-cli project, locate the file all-in-one.yaml

  6. For each container that you would like to deploy with custom changes, also update your Docker Hub repo info located in all-in-one.yaml for the changed microservice

  7. Update any references to Docker Hub in the individual service scripts in the remotemonitoring/individual folder that you would like to update

  • For example in iothub-manager.yaml, modify the following line to your own Docker Hub image repo
  • image: azureiotpcs/iothub-manager-dotnet:latest -> image: {your-docker-hub-username/{your-docker-hub-repo}:latest

Follow these steps to update containers

Update a deployed container via manual copy

If you would like to push your changes without creating a Docker Hub account you can follow these steps.
Note: all of the setup scripts on the VM reply on docker hub repositories, so you will need to manually start the service every time you wish to update.

  1. Generate a new docker image for your modified project
  2. cd {your-repo}/scripts/docker
  3. Build Docker image \
    • Windows cmd prompt:build
    • bash: ./build
  4. Copy your docker image to your VM using SCP. (more info on SCP to an Azure VM here)
    1. Get image id with docker images 
    docker save <image-id> > <path-to-output-file>/savedImage.tar
scp savedImage.tar <vm-username>@<vm-ip>:savedImage.tar
  5. SSH into the VM
  6. from vm, check to make sure simulation.tar exists with ls
  7. docker load < simulation.tar
  8. Find the new image id with docker images
  9. Edit the docker-compose.yml file to replace the docker-hub image with your image ID
    1. sudo nano /app/docker-compose.yml
    2. modify the desired container with your container id
      devicesimulation:
image: <your-image-id-here>
depends_on:
  - storageadapter
environment:
  - PCS_IOTHUB_CONNSTRING
  - PCS_STORAGEADAPTER_WEBSERVICE_URL=http://storageadapter:9022/v1
  - PCS_AUTH_ISSUER
  - PCS_AUTH_AUDIENCE
  - PCS_AUTH_REQUIRED
  - PCS_CORS_WHITELIST
  - PCS_APPLICATION_SECRET
#volumes:
#  - ./sample-volume:/app/data:ro
  10. run the start script to start the new container with the proper environment variables using the newly edited docker-compose.yml file. 
    ./start.sh
  11. verify your new image is running with docker ps -a

Local Development

Prerequisites

Running the PCS Web UI against deployed services

If you have your pcs up and running and would like to customize the UI, you can start implementing against deployed services by updating the endpoint listed in the .env file.

REACT_APP_BASE_SERVICE_URL={your-endpoint-url-from-pcs-cli}

More info in the README.MD for the pcs-remote-monitoring-webui

Running all PCS microservices locally

Running services with Docker

  1. Clone the pcs-webui project
  2. cd pcs-remote-monitoring-webui-scripts-localhost
  3. Install nginx
  4. Configure the environment variables used in docker-compose.yml
  5. Start the React application with npm start
  6. Run the start script: ./start.sh
  7. Visit http://127.0.0.1:9000

Running services locally

  1. Clone all of the repositories in this clickable map: .NET Repo Map Java Repo Map

  2. One-by-one, follow the README setup instructions to run a service locally. Typically this requires the following:

  • put any required resource information in the enviornment variables setup script located in each repo in {repo}/scripts/env-vars-setup
    • this may require you to know information like your DocumentDB connection string. See each repo's individual README.md for more info.
  • service dependencies:
    • if you don't want to have to spin up a local instance of a service and you have a running deployment, you can swap out {service}_WEBSERVICE_URL with the DNS endpoint for the running service
  • run the env-vars-setup script for your environment: ./env-vars-setup or env-vars-setup for Windows
  • run the service
    • Locally - use the instructions for your IDE in the README.md for your repo
    • With Docker

Customization

Changing container names

If you would like to change the name of your containers, you will need to modify the name of the container in the pcs-cli files for:

  • all-in-one.yaml
  • the remotemonitoring/scripts/individual file for the container
  • ingress.yaml
  • deployment-configmap.yaml

After you have renamed the containter in the pcs-cli, you will need to redeploy with the pcs-cli

Upgrade map key to see devices on a dynamic map

Switch to a static map

Clone this wiki locally