> For the complete documentation index, see [llms.txt](https://code.janardhanpulivarthi.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://code.janardhanpulivarthi.com/google-cloud/cloud-logging.md). # Cloud logging Log library . . ![](/files/-MSS7TethRniuiuPWVXj) Creating a sink ![](/files/-MSS83xZ-gjzxOIBdllw) Create logs routing sink ![](/files/-MSS8eYj6Orv0dyDqxYd) Apigee script ```bash # rewind.sh # Apigee AppMod Rewind Script set -e echo "====Starting Apigee Rewind Script====" # [ -z "$APIGEE_USER" ] && { echo "Need to set APIGEE_USER"; exit 1; } # [ -z "$APIGEE_PW" ] && { echo "Need to set APIGEE_PW"; exit 1; } # [ -z "$APIGEE_ORG" ] && { echo "Need to set APIGEE_ORG"; exit 1; } if [ -z "$APIGEE_ORG" ] then echo "Enter orgname:" read APIGEE_ORG fi if [ -z "$APIGEE_USER" ] then echo "Enter username:" read APIGEE_USER fi if [ -z "$APIGEE_PW" ] then echo "Enter password:" read -s APIGEE_PW fi APIGEE_ENV=test PRODUCT_NAME=lab04-hipster-products APP_NAME=lab04-hipster-app DEV_EMAIL=alice.smith@example.com PROXY_NAME=lab04-products EXT_NAME=goauthext GCP_PROJECT_ID=apigee-quest-v2 # clean up extensions echo "" echo -n "This lab will delete all existing Apigee Extensions in your Org. Please confirm: [y]" read EXT_CONFIRM if [ "$EXT_CONFIRM" != "y" ] then echo "Script aborted. To proceed with this lab, please re-run the script and choose 'y' when prompted." exit fi token_json=$(curl --fail -d "grant_type=password" --data-urlencode "username=${APIGEE_USER}" \ --data-urlencode "password=${APIGEE_PW}" https://login.apigee.com/oauth/token \ -H "Content-Type: application/x-www-form-urlencoded;charset=utf-8" \ -H "accept: application/json;charset=utf-8" -H "Authorization: Basic ZWRnZWNsaTplZGdlY2xpc2VjcmV0") TOKEN=$(jq -r '.access_token' <<< "$token_json") if [ -z "$TOKEN" ] then echo "Token was not generated. Script exiting" exit fi echo "====Deleting extensions====" curl -X DELETE -H "Authorization: Bearer ${TOKEN}" \ "https://api.enterprise.apigee.com/v1/organizations/${APIGEE_ORG}/environments/${APIGEE_ENV}/extensions" echo "====Cleanup completed====" # retrieve extension package info extpkg=$(curl -X GET --fail -H "Authorization: Bearer ${TOKEN}" "https://api.enterprise.apigee.com/extensionpackages?name=gcp-auth") pkgversion=$(jq -r '.contents[0].packages[0].version' <<< "${extpkg}") # Create the Google Oauth Extension echo "====Creating extension: ${EXT_NAME}====" ext=$(curl -X POST --fail -H "Authorization: Bearer ${TOKEN}" -H "Content-Type: application/json" -d "{ \"name\" : \"${EXT_NAME}\", \"packageName\": \"gcp-auth\", \"version\" : \"${pkgversion}\", \"description\" : \"Google Oauth extension for BigQuery api\" }" \ "https://api.enterprise.apigee.com/v1/organizations/${APIGEE_ORG}/environments/${APIGEE_ENV}/extensions") if [ -z "$ext" ] then echo "Failed creating extension. Script exiting" exit fi # Retrieve the extension ID extSelfUrl=$(jq -r '.self' <<< ${ext}) extId=${extSelfUrl##*/} if [ -z "$extId" ] then echo "Failed creating extension. Script exiting" exit fi echo "====Extension Created====Extension ID: $extId" echo "Follow the instuctions in the lab to now deploy the extension to your Apigee environment." ``` Logs sink to BigQuery ![](/files/-MSSFi30m9dR1VNNns2g) Log Explorer ![error](/files/-MWjXsTiC7bvJY0k5lJy) Log entry ```javascript { "protoPayload": { "@type": "type.googleapis.com/google.cloud.audit.AuditLog", "authenticationInfo": { "principalEmail": "service-1056631696419@container-engine-robot.iam.gserviceaccount.com" }, "requestMetadata": { "callerIp": "2002:a05:6624:104b::", "callerSuppliedUserAgent": "google-api-go-client/0.5 GoogleContainerEngine/v1" }, "serviceName": "compute.googleapis.com", "methodName": "v1.compute.firewalls.insert", "resourceName": "projects/qwiklabs-gcp-00-b04eb3d0f271/global/firewalls/gke-gke-demo-cluster-633239e5-master", "request": { "@type": "type.googleapis.com/compute.firewalls.insert" } }, "insertId": "epgof7dbp14", "resource": { "type": "gce_firewall_rule", "labels": { "project_id": "qwiklabs-gcp-00-b04eb3d0f271", "firewall_rule_id": "4147128468958439599" } }, "timestamp": "2021-04-22T01:49:55.417839Z", "severity": "NOTICE", "logName": "projects/qwiklabs-gcp-00-b04eb3d0f271/logs/cloudaudit.googleapis.com%2Factivity", "operation": { "id": "operation-1619056192549-5c085e53a719f-d4ea3bda-c5906a97", "producer": "compute.googleapis.com", "last": true }, "receiveTimestamp": "2021-04-22T01:49:55.737530398Z" } ``` ## Monitoring trace ![](/files/-MYlUnKItEST5uNRrsxk) ![](/files/-MYlUg_yb5csKIZlFc7g) ![](/files/-MWjY4eBFDzgIrBCN0NL) Locust * Request statistics * Response time distribution * Failures * Exceptions ![](/files/-M_EseclL5leizb-_lst) ![](/files/-MWjZWq7ea4U8dEJluqq) Corresponding VM usage and locust requests per second ![](/files/-M_Exr0TcHPFytKbMIrG) ![](/files/-M_ExwC4dtsTJsIrCZa-) ## Analysis of logs in BigQuery Refer: VPC flow logs - Analyzing network traffic. ```sql #standardSQL SELECT jsonPayload.src_vpc.vpc_name, SUM(CAST(jsonPayload.bytes_sent AS INT64)) AS bytes, jsonPayload.src_vpc.subnetwork_name, jsonPayload.connection.src_ip, jsonPayload.connection.src_port, jsonPayload.connection.dest_ip, jsonPayload.connection.dest_port, jsonPayload.connection.protocol FROM `qwiklabs-gcp-00-29c7708f8162.bq_vpcflows.compute_googleapis_com_vpc_flows_20210330` GROUP BY jsonPayload.src_vpc.vpc_name, jsonPayload.src_vpc.subnetwork_name, jsonPayload.connection.src_ip, jsonPayload.connection.src_port, jsonPayload.connection.dest_ip, jsonPayload.connection.dest_port, jsonPayload.connection.protocol ORDER BY bytes DESC LIMIT 15 ``` response: ![](/files/-MX3cSKwcsFSBOONcWHI) ## How to use terraform? {% code title="tracing-demo-deployment.yml" %} ```yaml apiVersion: apps/v1 kind: Deployment metadata: name: tracing-demo labels: app: tracing-demo spec: replicas: 1 selector: matchLabels: app: tracing-demo template: metadata: labels: app: tracing-demo spec: containers: - name: tracing-demo-container image: gcr.io/pso-examples/tracing-demo:1.0.0 ports: - containerPort: 8080 --- apiVersion: v1 kind: Service metadata: name: tracing-demo spec: selector: app: tracing-demo ports: - protocol: TCP port: 80 targetPort: 8080 type: LoadBalancer ``` {% endcode %} ![](/files/-MYlZfRps-DVuC0J6chx) ![](/files/-MYlZAsKRr6W7Up_46XN) ![](/files/-MYlZGbc5fnby23viAn4) ![](/files/-MYlZbS2fmembJUGjRjB) ![](/files/-MYlZWhQVUX-SKrXbFY1) Permission ![](/files/-MYr0IH6N5bfefuFUjfC) ![](/files/-MYr0NuS3n0gJtWcUYFH)