# Cloud logging

Log library

.<https://cloud.google.com/logging/docs/view/query-library>

.<https://cloud.google.com/logging/docs/view/query-library-preview>

![](/files/-MSS7TethRniuiuPWVXj)

Creating a sink

![](/files/-MSS83xZ-gjzxOIBdllw)

Create logs routing sink

![](/files/-MSS8eYj6Orv0dyDqxYd)

Apigee script

```bash
# rewind.sh
# Apigee AppMod Rewind Script
set -e

echo "====Starting Apigee Rewind Script===="
# [ -z "$APIGEE_USER" ] && { echo "Need to set APIGEE_USER"; exit 1; }
# [ -z "$APIGEE_PW" ] && { echo "Need to set APIGEE_PW"; exit 1; }
# [ -z "$APIGEE_ORG" ] && { echo "Need to set APIGEE_ORG"; exit 1; }

if [ -z "$APIGEE_ORG" ]
  then
    echo "Enter orgname:"
    read APIGEE_ORG
fi
if [ -z "$APIGEE_USER" ]
  then
    echo "Enter username:"
    read APIGEE_USER
fi
if [ -z "$APIGEE_PW" ]
  then
    echo "Enter password:"
    read -s APIGEE_PW
fi

APIGEE_ENV=test
PRODUCT_NAME=lab04-hipster-products
APP_NAME=lab04-hipster-app
DEV_EMAIL=alice.smith@example.com
PROXY_NAME=lab04-products
EXT_NAME=goauthext
GCP_PROJECT_ID=apigee-quest-v2

# clean up extensions
echo ""
echo -n "This lab will delete all existing Apigee Extensions in your Org. Please confirm: [y]"
read EXT_CONFIRM
if [ "$EXT_CONFIRM" != "y" ]
  then
    echo "Script aborted. To proceed with this lab, please re-run the script and choose 'y' when prompted."
    exit
fi

token_json=$(curl --fail -d "grant_type=password" --data-urlencode "username=${APIGEE_USER}" \
  --data-urlencode "password=${APIGEE_PW}" https://login.apigee.com/oauth/token \
  -H "Content-Type: application/x-www-form-urlencoded;charset=utf-8" \
  -H "accept: application/json;charset=utf-8" -H "Authorization: Basic ZWRnZWNsaTplZGdlY2xpc2VjcmV0")

TOKEN=$(jq -r '.access_token' <<< "$token_json")
if [ -z "$TOKEN" ]
  then
    echo "Token was not generated. Script exiting"
    exit
fi

echo "====Deleting extensions===="
curl -X DELETE -H "Authorization: Bearer ${TOKEN}" \
 "https://api.enterprise.apigee.com/v1/organizations/${APIGEE_ORG}/environments/${APIGEE_ENV}/extensions"

echo "====Cleanup completed===="

# retrieve extension package info
extpkg=$(curl -X GET --fail -H "Authorization: Bearer ${TOKEN}" "https://api.enterprise.apigee.com/extensionpackages?name=gcp-auth")
pkgversion=$(jq -r '.contents[0].packages[0].version' <<< "${extpkg}")

# Create the Google Oauth Extension
echo "====Creating extension: ${EXT_NAME}===="
ext=$(curl -X POST --fail -H "Authorization: Bearer ${TOKEN}" -H "Content-Type: application/json" -d "{
 \"name\" : \"${EXT_NAME}\",
 \"packageName\": \"gcp-auth\",
 \"version\" : \"${pkgversion}\",
 \"description\" : \"Google Oauth extension for BigQuery api\" }" \
 "https://api.enterprise.apigee.com/v1/organizations/${APIGEE_ORG}/environments/${APIGEE_ENV}/extensions")

if [ -z "$ext" ]
  then
    echo "Failed creating extension. Script exiting"
    exit
fi

# Retrieve the extension ID
extSelfUrl=$(jq -r '.self' <<< ${ext})
extId=${extSelfUrl##*/}

if [ -z "$extId" ]
  then
    echo "Failed creating extension. Script exiting"
    exit
fi

echo "====Extension Created====Extension ID: $extId"
echo "Follow the instuctions in the lab to now deploy the extension to your Apigee environment."
```

Logs sink to BigQuery

![](/files/-MSSFi30m9dR1VNNns2g)

Log Explorer

![error](/files/-MWjXsTiC7bvJY0k5lJy)

Log entry

```javascript
{
  "protoPayload": {
    "@type": "type.googleapis.com/google.cloud.audit.AuditLog",
    "authenticationInfo": {
      "principalEmail": "service-1056631696419@container-engine-robot.iam.gserviceaccount.com"
    },
    "requestMetadata": {
      "callerIp": "2002:a05:6624:104b::",
      "callerSuppliedUserAgent": "google-api-go-client/0.5 GoogleContainerEngine/v1"
    },
    "serviceName": "compute.googleapis.com",
    "methodName": "v1.compute.firewalls.insert",
    "resourceName": "projects/qwiklabs-gcp-00-b04eb3d0f271/global/firewalls/gke-gke-demo-cluster-633239e5-master",
    "request": {
      "@type": "type.googleapis.com/compute.firewalls.insert"
    }
  },
  "insertId": "epgof7dbp14",
  "resource": {
    "type": "gce_firewall_rule",
    "labels": {
      "project_id": "qwiklabs-gcp-00-b04eb3d0f271",
      "firewall_rule_id": "4147128468958439599"
    }
  },
  "timestamp": "2021-04-22T01:49:55.417839Z",
  "severity": "NOTICE",
  "logName": "projects/qwiklabs-gcp-00-b04eb3d0f271/logs/cloudaudit.googleapis.com%2Factivity",
  "operation": {
    "id": "operation-1619056192549-5c085e53a719f-d4ea3bda-c5906a97",
    "producer": "compute.googleapis.com",
    "last": true
  },
  "receiveTimestamp": "2021-04-22T01:49:55.737530398Z"
}
```

## Monitoring

trace

![](/files/-MYlUnKItEST5uNRrsxk)

![](/files/-MYlUg_yb5csKIZlFc7g)

![](/files/-MWjY4eBFDzgIrBCN0NL)

Locust

* Request statistics
* Response time distribution
* Failures
* Exceptions

![](/files/-M_EseclL5leizb-_lst)

![](/files/-MWjZWq7ea4U8dEJluqq)

Corresponding VM usage and locust requests per second

![](/files/-M_Exr0TcHPFytKbMIrG)

![](/files/-M_ExwC4dtsTJsIrCZa-)

## Analysis of logs in BigQuery

Refer: <https://google.qwiklabs.com/focuses/1236?parent=catalog> VPC flow logs - Analyzing network traffic.

```sql
#standardSQL
SELECT
jsonPayload.src_vpc.vpc_name,
SUM(CAST(jsonPayload.bytes_sent AS INT64)) AS bytes,
jsonPayload.src_vpc.subnetwork_name,
jsonPayload.connection.src_ip,
jsonPayload.connection.src_port,
jsonPayload.connection.dest_ip,
jsonPayload.connection.dest_port,
jsonPayload.connection.protocol
FROM
`qwiklabs-gcp-00-29c7708f8162.bq_vpcflows.compute_googleapis_com_vpc_flows_20210330`
GROUP BY
jsonPayload.src_vpc.vpc_name,
jsonPayload.src_vpc.subnetwork_name,
jsonPayload.connection.src_ip,
jsonPayload.connection.src_port,
jsonPayload.connection.dest_ip,
jsonPayload.connection.dest_port,
jsonPayload.connection.protocol
ORDER BY
bytes DESC
LIMIT
15
```

response:

![](/files/-MX3cSKwcsFSBOONcWHI)

## How to use terraform?

{% code title="tracing-demo-deployment.yml" %}

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: tracing-demo
  labels:
    app: tracing-demo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: tracing-demo
  template:
    metadata:
      labels:
        app: tracing-demo
    spec:
      containers:
      - name: tracing-demo-container
        image: gcr.io/pso-examples/tracing-demo:1.0.0
        ports:
        - containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: tracing-demo
spec:
  selector:
    app: tracing-demo
  ports:
  - protocol: TCP
    port: 80
    targetPort: 8080
  type: LoadBalancer
```

{% endcode %}

![](/files/-MYlZfRps-DVuC0J6chx)

![](/files/-MYlZAsKRr6W7Up_46XN)

![](/files/-MYlZGbc5fnby23viAn4)

![](/files/-MYlZbS2fmembJUGjRjB)

![](/files/-MYlZWhQVUX-SKrXbFY1)

Permission

![](/files/-MYr0IH6N5bfefuFUjfC)

![](/files/-MYr0NuS3n0gJtWcUYFH)


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://code.janardhanpulivarthi.com/google-cloud/cloud-logging.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.