Assessment questions

Data Engineering

Question 1

Question 2

.https://cloud.google.com/solutions/migration/hadoop/hadoop-gcp-migration-overview

Question 3

Question 4

Question 5

Use Cloud Spanner for storage. Monitor CPU utilization and increase node count if more than 70% utilized for your time span

Question 6

Question 7

.https://cloud.google.com/dataflow/docs/concepts/streaming-pipelines#session-windows

.https://cloud.google.com/spanner/docs/transactions?hl=en

Question 8

// BigtableIO
// https://beam.apache.org/releases/javadoc/2.1.0/org/apache/beam/sdk/io/gcp/bigtable/BigtableIO.html

BigtableOptions.Builder optionsBuilder =
     new BigtableOptions.Builder()
         .setProjectId("project")
         .setInstanceId("instance");

 PCollection<KV<ByteString, Iterable<Mutation>>> data = ...;

 data.apply("write",
     BigtableIO.write()
         .withBigtableOptions(optionsBuilder)
         .withTableId("table"));