main
  • About
  • Civil Engineering
    • Interview questions
    • Bridge design
  • Google Cloud
    • Code samples
    • kafka
    • Cloud Run
    • persistent disks
    • Spinnaker
    • Assessment questions
    • IAM
    • Cloud Storage
    • VPC
    • Cost optimization
    • Compute Engine
    • App Engine
    • Cloud Vision
    • Spanner
    • Cloud SQL
    • Solutions
      • Static IP - WIP
      • Network performance
      • Building a VPN
      • Build a streaming app
      • ML train with taxi data
    • Dataproc
    • Dataprep
    • BigTable
    • Cloud Fusion
    • Data flow
    • CloudFront
    • APIGEE
    • BigQuery
    • Cloud logging
    • Pubsub
    • Identity Aware Proxy
    • Data center migration
    • Deployment Manager
    • Kubeflow
    • Kubernetes Engine
    • Istio
    • Read the following
    • Storage for cloud shell
    • kms
    • kpt
    • Hybrid cloud with Anthos
    • helm
    • Architecture
    • terraform
    • Network
    • Data studio
    • Actions
    • Jenkins
  • Data Processing
    • Data Lake
    • Data ingestion
    • Data Cleaning - Deduplication
    • Data Cleaning - Transformation
    • Data cleaning - rule definition
    • ETL
  • Machine Learning
    • Tensorflow
    • Tensorflow tips
    • Keras
    • Scikit-learn
    • Machine learning uses
    • Working with Pytorch
    • Federated learning
  • AWS cloud
    • Billing
    • Decrease volume size of EC2
    • Run CVE search engine
    • DataSync
    • EC2 spot instances
  • Java
    • Java
    • NIO
    • System Design
      • Zero trust framework
    • Collections
  • Azure
    • Enterprise Scale
    • API
    • Resource group
    • Create an sql database
  • UBUNTU
    • No Release file
    • STRATO blockchain
    • iperf
    • Rsync
    • curl
    • Shell
    • FAQ - git
  • PH test
    • Syllabus
    • Opportunities
    • Aptitude test
  • Development
    • Course creation
    • web.dev
    • docfx template
  • npm
  • Docker Desktop
  • Nginx
  • English rules
  • Confluent
  • sanity theme
  • Java Native Interface tutorial
  • Putty
  • Personal website host
  • Google search SEO
  • Reading a textbook
  • DFCC Progress
  • STORAGE
    • Untitled
  • Services Definition
    • Cloud VPN and routing
  • Microservices design and Architecture
    • Untitled
  • Hybrid network architecture
    • Untitled
  • Deployment
    • Untitled
  • Reliability
    • Untitled
  • Security
    • Untitled
  • Maintenance and Monitoring
    • Peering
  • Archive
    • parse dml to markdown
Powered by GitBook
On this page

Was this helpful?

  1. Google Cloud

Read the following

A comprehensive reading is required of the following

PreviousIstioNextStorage for cloud shell

Last updated 4 years ago

Was this helpful?

.

.

Hive and Dataproc and how it works on Google cloud

How the workers can download dependencies from internet.

How Cloud Datastore and Spanner are different

When to use spanner vs Bigtable

2 PB where will you store -> Datastore, spanner, Bigtable

1 TB where would you store

KafkaIO

Storage transfer service vs transfer appliance. Can you use a private web address to transfer 2 PB of data over six months with Storage transfer service?

Migrating from on-premises to GCP. Fault tolerant method?

  • Do you copy the files to GCS and run dataproc and changes references hdfs:// to gs://?

Partitioning for each export timestamp to decrease the query bill? (timestamp, ID are used for input)

Data analysts are querying last 30 days more. You want to decrease cost immediately. Data arrives once each day?

  • copy the last 30 files to gcs do analysis over them?

  • Export 30 days data to Datastudio?

  • create a table/partition with last 30-60 days data?

Dataflow template vs DAG on composer for running spark some of the jobs in sequences and some concurrent?

AUC under curve 1.05. How would you improve the Area under the curve?

fluentd in_tail plugin vs Mysql plugin for MariaDB with Stack driver agent?

Cloud vision API have damage detection without training?

Cloud ML vs Dataproc spark ML from existing spark ML models. and Can we pull data from BigQuery directly?

Increase availability: For SQL like querying, storage for over years, streaming

  • Add data in GCS in multi-region. Pull data by BigQuery, Dataflow, Cloud storage

Pubsub

Kubeflow

https://cloud.google.com/bigquery/docs/reference/standard-sql/analytic-function-concepts
https://cloud.google.com/bigquery/streaming-data-into-bigquery
https://cloud.google.com/solutions/using-apache-hive-on-cloud-dataproc