Technical articles from Pivotal engineers.
-
Benchmarking Apache Geode's Performance
Measuring and Improving Performance for a Faster Geode 1.10
Categories:
Performance
Apache Geode
AWS
Java
Testing
-
How we built a controller using KubeBuilder with test-driven development, Part 2
Lessons we learned while building a controller with KubeBuilder, and how we
made our unit tests fast enough for test driving.
Categories:
Greenplum
Kubernetes
KubeBuilder
Test Driven Development
-
How we built a controller using KubeBuilder with Test Driven development, Part 1
Lessons we learned while building a controller with KubeBuilder
Categories:
Greenplum
Kubernetes
KubeBuilder
-
ESM Study Data
Anonymized Data from a research study.
Categories:
Research
Agile
-
Merge Join Support In GPORCA
We look at how FULL JOINS can be supported natively in GPORCA, allowing GPORCA to generate plans providing better query execution times.
Categories:
Databases
Query Optimization
SQL
Greenplum
GPORCA
Full Join
-
Greenplum: Speeding up JDBC Reads in PXF Using Partitioning
Parallel data reads from external databases in PXF using JDBC Partitioning
Categories:
Greenplum
PXF
JDBC
Partitioning
Parallel
JDBC Reads
-
Administering a PXF cluster using the PXF Command Line Interface
The PXF CLI provides a convenient way to administer PXF servers running on Greenplum
Categories:
Greenplum
GPDB
PXF
CLI
Utilities
Amazon S3
HDFS
-
Fast Access to Your S3 Data with Greenplum PXF
Greenplum's Platform Extension Framework (PXF) supports querying data on AWS S3. PXF now supports AWS S3 Select which will enable Greenplum users to run their queries on data stored in AWS S3 more efficiently with up to 4x performance improvements.
Categories:
Greenplum Database
GPDB
Platform Extension Framework
PXF
S3 Select
AWS
Amazon S3
Amazon Simple Storage Service
-
Ensure Windows apps are using the `windows` stack before upgrading to PASW 2.8
Pivotal Application Service for Windows introduced the
-s windows
stack name in PASW 2.4, reducing the operator and developer need to concern themselves with specific Windows Server versions. From PASW 2.4 thru PASW 2.7, both the windows2016
and windows
stack names worked - giving sufficient time to migrate apps over to the new stack name.
Categories:
PASW
.NET
Windows
stack
cf
-
A High-performing Mid-range NAS Server, Part 3: 10 GbE
Upgrading our iSCSI (Internet Small Computer System Interface) NAS
(network-attached storage) server from 1 GbE (gigabit ethernet) to 10 GbE
increased our vSphere VM's (Virtual Machine's) sequential read throughput 43%,
sequential write throughput 940% (!), and IOPS (Input/Output Operations Per
Second) 60%.
Categories:
Logging & Metrics
FreeNAS
-
How we moved a massively parallel Postgres database onto Kubernetes
Distributed applications that scale out, such as Greenplum, fit well with Kubernetes.
Categories:
Greenplum
Kubernetes
PostgreSQL
Databases
-
Testing Spring filters without pain
This article aims at showing you how to test http filters effectively, by avoiding unnecessary mocking and rather appropriately using the test toolkit that Spring offers.
Categories:
Spring
TDD
Mocking
Kotlin
-
Eureka, Zuul, and Cloud Configuration - Pivotal Cloud Foundry
Example and explanation of how to set up a common use case with the Spring Cloud Netflix stack within a PCF deployment.
Categories:
Java
Spring
Annotations
Spring Boot
Pivotal Cloud Foundry
-
Eureka, Zuul, and Cloud Configuration - Local Development
Example and explanation of how to set up a common use case with the Spring Cloud Netflix stack to prototype for local development.
Categories:
Java
Spring
Annotations
Spring Boot
-
Using Greenplum to access Minio distributed object storage server
Parallel data transfer data between Greenplum and Minio
Categories:
Greenplum
Minio
PXF
-
Transferring Time-based One-time Passwords to a New Smartphone
Time-based One-time Passwords (TOTP) authenticator apps are often deployed on
smartphones to enhance security of sensitive online accounts; however,
replacing the phone typically requires a burdensome reset of all TOTP
passwords. In this blog post we describe the clever use of a QR code reader,
secure storage, bash scripting, and a QR code generator to quickly configure a
new phone (no reset of TOTP required).
Categories:
2FA
TOTP
Google Authenticator
-
Managing Stateful Apps with the Operator Pattern; Orchestration Considerations
Stateful Kubernetes apps can be managed with the Operator Pattern, and have other considerations regarding their orchestration; the forth blog of a series of 4 on Stateless Kubernetes Apps
-
Storing Stateful Data that Outlives a Container or a Cluster; Optimizing for Local Volumes
Storage for the data of stateful Kubernetes apps must outlive a Kubernetes container or a cluster, and can be optimized with Local Volumes; the third blog of a series of 4 on Stateless Kubernetes Apps
-
Provisioning Stateful Kubernetes Containers that Work Hard and Stay Alive
How to develop Kubernetes containers that resist eviction, the second blog of a series of 4 on Stateless Kubernetes Apps
-
Stateful Apps in Kubernetes
Defining stateful versus stateless applications in terms of Kubernetes service resiliency. The first blog post of a series of four about Stateful Kubernetes Apps.
-
Troubleshooting Obscure OpenSSH Failures
How an elusive CI (Continuous Integration) error led us to uncover a hidden man-in-the-middle ssh proxy.
Categories:
Networking
-
Greenplum for Kubernetes Operator
Enabling Declarative Style Greenplum Deployment in Kubernetes
Categories:
Greenplum
Kubernetes
Agile
-
Understanding Keep-Alive Timeouts in the Cloud Foundry Networking Stack
Let's journey through a debugging session to understand best practices in configuring the Cloud Foundry deployment's networking stack, debug interacting keep-alive timeouts, and get familiar with the TCP layer.
Categories:
Networking
Routing
Cloud Foundry
Keep-Alives
Debugging
TCP
-
Quoted and Unquoted Template Variables in Concourse CI
How to use quoted and unquoted variables in Concourse CI templates
Categories:
Concourse
-
Development of Greenplum for Kubernetes using Minikube
Speed up agile developing process using a minikube
Categories:
Greenplum
Kubernetes
Agile
-
Concourse as a Service and Multi-CPI
Leveraging a singular BOSH director to manage Concourse for multiple teams
Categories:
BOSH
Concourse
GCP
Infrastructure
-
Let's Contribute to Golang!
How to start contributing to the Golang project with an example
Categories:
Golang
contributing
OSS
-
Safely Upgrading PAS 2.2 with NSX-T Load Balancers
How to upgrade an NSX-T-based PAS 2.2 → 2.3 foundation without downtime
Categories:
NSX-T
Operations Manager
BOSH
-
GiST Support In GPORCA
We look at how GIST indexes can be supported in GPORCA, allowing GPORCA to generate plans providing better query execution times.
Categories:
Databases
Query Optimization
SQL
Greenplum
GPORCA
GiST
-
Frontend Contract Tests Without Magic Numbers
Banish magic numbers from your frontend contract tests!
Categories:
Spring
API
Contract Testing
-
Let's use Vault - Part 3: Setting up Vault
How to configure Vault for your CI
Categories:
GCP
Kubernetes
Vault
Credhub
-
Let's use Vault - Part 2: Setting up Vault
How to configure Vault for your team
Categories:
GCP
Kubernetes
Vault
Credhub
-
Let's use Vault - Part 1: Deploying Vault
How to deploy Vault on Kubernetes using Google Cloud Storage as its backend
Categories:
GCP
Kubernetes
Vault
Credhub
-
Diagnosing Ruby Memory Issues in Cloud Foundry's API Server
How the Cloud Foundry CAPI team debugged Ruby memory usage issues in the Cloud Controller API
Categories:
Cloud Foundry
CF
Cloud Controller
Ruby
Ruby Memory Management
-
How to use "tag_filter" in "git" resources in Concourse CI
How to use “tag_filter” in “git” resources in Concourse CI
Categories:
git
GitHub
Concourse
-
How to Install a TLS Certificate on vCenter Server Appliance (VCSA) 6.7
We install a Transport Layer Security (TLS) certificate issued by a
commercial Certificate Authority (CA) on a VMware VCSA 6.7 while avoiding
several pitfalls.
Categories:
vSphere
-
How Pivotal contributes to the Development of PostgreSQL 11
Summary of Pivotal Contributions to PostgreSQL 11
Categories:
Greenplum Database
PostgreSQL
Greenplum
Development
-
Create Regression Tests for Greenplum Database
How to create new regression tests for Greenplum Database
Categories:
Greenplum Database
Databases
Testing
Regression Tests
-
Windows Containers in Cloud Foundry? Here's How We Did It
Technical details for how we implemented containers on Windows Server Core 2016 for Pivotal
Cloud Foundry.
Categories:
CF Runtime
Diego
Windows
Containers
.NET
-
Scaling Machine Learning to Recommend Driving Routes
Categories:
Spark & Hadoop
real-time prediction & recommendation
feature engineering & modeling pipeline
driver revenue prediction
Scale Machine Learning as a service
-
Failure is a part of the game
Failure is necessary
-
Benchmarking the Disk Speed of IaaSes
It's helpful to know the performance characteristics when selecting a disk
type for an Infrastructure as a Service (IaaS). In this blog post we describe
the results of benchmarking various disk types of various IaaSes, including
Amazon Web Services (AWS), Microsoft Azure, Google Compute Engine (GCE), and
VMware vSphere. We measure Input/output operations per second (IOPS), read
throughput, and write throughput.
Categories:
BOSH
Logging & Metrics
-
Event Sourcing with Kafka, RabbitMQ and JPA
Demonstrating Event Sourcing with various methods using Kafka, RabbitMQ and JPA
Categories:
Event Sourcing
Kafka
RabbitMQ
JPA
Spring Cloud Stream
-
Simplify Deploying Kubernetes with Cloud Foundry Container Runtime
Cloud Foundry Container Runtime makes deploying Kubernetes easy with the power of BOSH.
Categories:
CFCR
BOSH
Tutorial
Kubernetes
DevOps
-
Deploying BOSH VMs with IPv6 Addresses on vSphere
Recent changes to the BOSH software suite enable the assignment of IPv6
addresses to VMs deployed by the BOSH Director in a vSphere environment.
In this blog post we describe how we deployed a BOSH Director and
subsequently used the Director to deploy a web server with a private
IPv4 address and a public IPv6 address.
Categories:
BOSH
IPv6
-
Diagnosing performance issues with Java legacy apps on Pivotal Cloud Foundry
How to profile apps on Pivotal Cloud Foundry. VisualVM to the rescue!
Categories:
Profiling
Java
VisualVM
Performance
CPU Sampling
-
All I do is VIM VIM VIM
A subset of useful vim commands for beginners
Categories:
vim
-
Maintaining BOSH Directors with Concourse CI and bosh-deployment
BOSH directors can be deployed (and redeployed) with a properly built
Concourse pipeline and manifests generated by the curated
bosh-deployment
manifest generation tool. This lightens
the burden of keeping BOSH directors patched & up-to-date.
Categories:
BOSH
Concourse
-
Injecting Credentials Into Spring Boot Apps – The Easy Way
Configuring Spring Boot apps in Cloud Foundry is super easy once we take advantage of ConfigurationProperties and user-provided services.
Categories:
Spring Boot
Cloud Foundry
CF
-
React integration testing with Enzyme
We used Enzyme, a popular React unit testing library, for full-on frontend integration tests.
Find out how our setup greatly improves feedback and avoid the pitfalls we've come across.
Categories:
React
Enzyme
Javascript
Testing
-
Testing Greenplum Database using fault injection
How to test Greenplum Database using fault injection
Categories:
Greenplum Database
Greenplum
Testing frameworks
Testing
-
Continuous Market Disruption
This blog post explores how enterprises can reduce waste and continuously experiment leading to the product evolution and effective use of CAPEX to realize the true ROI Value of cloud native with Pivotal.
Categories:
App Transformation
Replatforming
Modernization
Pivotal Cloud Foundry
Cloud Native ROI Continuum
Agile
XP
-
Connecting To PCC with Spring Boot
How do you get started with Pivotal Cloud Cache (PCC)? How do you connect your spring boot application to PCC? This blog post attempts to provide you an example on how to connect your spring boot application to PCC.
Categories:
PCC
CF
-
Interpreting Decision Trees and Random Forests
This blog dives deeper into the fundamentals of decision trees and random forests to better interpret them.
Categories:
Decision Tree
Random Forest
Data Science
-
Transitioning to CredHub on Pivotal Web Services (PWS)
CredHub is designed to store passwords, keys, certificates, and other sensitive information for a BOSH-managed environment. Pivotal's Cloud Operations (CloudOps) team recently migrated credentials for PWS to CredHub. Here's how we did that.
Categories:
CredHub
Operations
Cloud Foundry
CF
-
Day in the Life - Santa Monica Style
A day in the life of a Pivotal Cloud Foundry software engineer in Santa Monica, California.
Categories:
BBL
Cloud Foundry
-
Deploying GrootFS to Pivotal Web Services (PWS)
Our rollout of GrootFS to Pivotal Web Services was a gradual, iterative process that allowed us to test on a small subset of production instances, roll the changes back, make improvements, and finally release it with confidence. We talk about the process and provide takeaways for other teams deploying new features to production.
Categories:
Operations
Rollout
Cloud Foundry
CF
Agile
-
Elixir Clustering And Discovery On Cloud Foundry
A tutorial for configuring an Elixir application to automatically cluster on Cloud Foundry (eg. PWS) using container to container networking.
Categories:
Elixir
Cloud Foundry
-
Deploying a BOSH Director With SSL Certificates Issued by Commercial CA
A BOSH director is typically deployed with self-signed SSL (Secure Sockets
Layer) certificates; however, the director can be deployed with certificates
issued by a trusted CA (Certificate Authority). Here's how.
Categories:
BOSH
-
Greenplum and Apache Spark via JDBC
Using Greenplum and Apache Spark via JDBC in 5 minutes
Categories:
Greenplum
Apache Spark
JDBC
Postgresql
-
Hibernate to JDBC
An exploration in replacing Hibernate with JDBC in Spring.
Categories:
Spring
Java
Hibernate
JDBC
Java Database Connectivity
Repositories
-
sql_magic: Jupyter Magic to Write SQL for Apache Spark and Relational Databases
An IPython library to help data scientists write SQL code
Categories:
Data Science
Jupyter Notebook
SQL
Greenplum
Apache Spark
-
Half a Million Concurrent/Distributed JMeter Users with BOSH ... in 10 Minutes
This post details the steps of utilizing BOSH to setup half a million (potentially much more) concurrent and distributed Apache JMeter threads for load testing.
Categories:
BOSH
Apache JMeter
Distributed Load Testing
Distributed Stress Testing
-
Using Terraform, Concourse, and om to Continuously Deploy Pivotal Cloud Foundry's Elastic Runtime
A look at how Release Engineering deploys and tests the Elastic Runtime Tile
Categories:
Infrastructure
Elastic Runtime Tile
Release Engineering
Terraform
Ops Manager
om
-
A Smart Editor and Visualizer for BOSH Manifests
BOSH Editor is in-browser application that facilitates the creation of BOSH manifests. It includes smart auto-complete features and a way to visualize different segments of the manifest on the fly.
Categories:
BOSH
Editor
Visualization
-
The Test Double Rule of Thumb
A guide to understanding and effectively utilizing test doubles.
Categories:
Agile
TDD
-
Deploying a Docker Registry to Cloud Foundry
Pushing a Docker Registry to Cloud Foundry can be a powerful approach
to distributing private Docker images to teams with minimal overhead.
Categories:
Docker
Cloud Foundry
-
Testing for Data Consistency on Cloud Cache using BOSH and Turbulence
Testing for data consistency on the Cloud Cache team using BOSH and Turbulence
Categories:
BOSH
Turbulence
Cloud Cache
-
dep Coming to Unify Package Management in Go
Golang's future standard package manager is becoming more usable everyday. Here's why it's necessary and how you can try it out today.
Categories:
Golang
Package Management
dep
-
Using Luigi Pipelines in a Data Science Workflow
This post shows how we use Luigi as a pipeline tool to manage a data science workflow. We walk through an example analyzing network traffic logs.
Categories:
Data Science
Luigi
Greenplum
Python
SQL
-
Must-Know Spring Boot Annotations: Controllers
Learn about the most essential, must-know annotations for Spring Boot controllers.
Categories:
Spring
Spring Boot
Annotations
MVC
-
Scoring at Scale with Keras and TensorFlow on Greenplum
How to train a deep neural network with Keras and TensorFlow and then apply it for scoring on Greenplum.
Categories:
Data Science
Greenplum
Greenplum Database
SQL
Python
-
The gpfdist protocol for External Tables in Greenplum Database
The internals of the gpfdist protocol used for External Tables in Greenplum Database
Categories:
Greenplum Database
Databases
-
Deploy To vSphere NSX-T Opaque Networks Using BOSH
BOSH now allows attaching vSphere deployed VMs to NSX-T's Opaque Networks
Categories:
BOSH
BOSH CPI
vSphere
NSX-T
-
Visualizing Cloud Foundry with Weave Scope - Part 1
This is the first of two blog posts showing you how Weave Scope, a visualization and troubleshooting tool originally aimed at Docker and Kubernetes, can be used to reveal the hosts and network topologies for arbitrary BOSH deployments. As such, Scope is able to visualize the many components that make up Cloud Foundry and the interplay between them. This gives everyone, from newcomers to seasoned experts, new ways to learn about Cloud Foundry and troubleshoot a running system if necessary.
Categories:
Cloud Foundry
CF
BOSH
Weave Scope
Visualization
Distributed Systems
-
Testing in Swift with Dependencies
Testing against frameworks/libraries is tricky in Swift because we can't just spy on dependencies and fake out the response. Here is how we test neatly in Swift 3.
Categories:
Swift
Swift 3
Protocol
Testing
-
Plotting Using an MPP Database
A tutorial on how to build histograms, scatter plots, and ROC curves using an MPP database and plot them in Python or R.
Categories:
HAWQ
GPDB
Big Data
PostgreSQL
Postgres
MPP
Histogram
Scatter Plot
ROC Curve
-
BOSH + Apache JMeter(TM) = Tornado for Apache JMeter(TM)
Tornado for Apache JMeter™, is a BOSH release for Apache JMeter™. It simplifies the usage of JMeter™ in distributed mode, making it easier to perform load or stress testing.
Categories:
BOSH
Apache JMeter
Distributed Load Testing
Distributed Stress Testing
-
Git pushInsteadOf
Best practices for Git on mixed-source teams
Categories:
git
-
Spring for Normal People
Learn how to Spring Boot the easy way with TDD and Thymeleaf. All the gain, half the pain!
Categories:
Spring
Spring Boot
TDD
Humans
-
TDD with React and MobX
A look into testing MobX + React, plus why MobX is a viable alternative to Redux.
Categories:
TDD
React
MobX
Redux
Javascript
-
Trilogy and Greenplum for Data Science TDD
How to use a new SQL testing framework called
Trilogy with Greenplum Database to help you
test drive your data science code.
Categories:
Data Science
TDD
SQL
Databases
Greenplum Database
-
Trilogy - the database testing framework
A quick overview of a new database-agnostic SQL testing framework
Categories:
SQL
TDD
Testing frameworks
Databases
-
Continuous Integration for Data Science
This article will show why continuous integration is also important for smart apps projects.
Categories:
Data Science
Machine Learning
Smart Apps
TDD
Continuous Integration
Concourse
-
Agile Development for Highly Scalable Data Processing Pipelines
Legacy data processing pipelines are slow, inaccurate, hard to debug, and can cause thousands of dollars in revenue. Conforming to agile methodology and a detailed seven-step approach can ensure an efficient, reliable and high-quality data pipeline on distributed data processing framework like Spark. Learn how following TDD, careful creation of data structures, and parallel execution results in: code competency and completeness, and a linearly or constantly scalable robust big data processing pipeline.
Categories:
Linear-Scale
Data-Pipeline
Agile-TDD
SPARK
Pair-programming
-
Why Is My NTP Server Costing $500/Year? Part 3
Running a Network Time Protocol (NTP) server in the pool.ntp.org project can
incur $500/year in data transfer (bandwidth) costs. Those costs can be reduced
or even eliminated by choosing alternative Infrastructure as a Service (IaaS)
providers, modifying the server's pool.ntp.org connection speed setting,
choosing an alternative continent upon which to place the server, and
modifying the NTP daemon's configuration file to rate-limit the clients.
Categories:
NTP
-
The File protocol for External Tables in Greenplum Database
The internals of the File protocol used for External Tables in Greenplum Database
Categories:
Greenplum Database
Databases
-
Enforcing Separation of Concerns with Declarative Programming
Using a popular academic program, FizzBuzz, and an extreme declarative programming approach using Datalog, we examine the thought process and benefits for enforcing separation of concerns with declarative programming.
Categories:
Declarative Programming
datalog
-
Understanding and Mitigating the .io Top-Level-Domain failure in Cloud Foundry and Pivotal Web Services
An article about the
.io
TLD failure, how it affected Cloud Foundry, as well as how we could potentially mitigate TLD failures in the future.
Categories:
CF Runtime
DNS
Cloud Foundry
Pivotal Web Services
-
Public IPs for diego cells
On September 29th, 2016, Pivotal Web Services (PWS) enabled a feature available in BOSH to auto-assign public IPs to Diego cells.
Categories:
BOSH
CF Runtime
-
Using the beta BOSH CLI to Deploy an IPv6-enabled nginx Server to AWS
Amazon Web Services (AWS) has recently announced Internet Protocol version 6
(IPv6) Support for their Elastic Compute Cloud (EC2) Instances in Virtual
Private Clouds (VPCs). In this blog post, we describe using the beta BOSH
command line interface (CLI) to deploy a virtual machine (VM) running nginx (a
popular webserver) to EC2 with a native IPv6 address.
Categories:
BOSH
IPv6
-
Using docker-image-resource to build a custom container for testing your Ruby apps in Concourse
Sometimes we want to create custom docker images with external dependencies cached. Learn how to have Concourse automate this and use the built container to run tests.
Categories:
Concourse
Docker
Ruby
Containerization
-
Cloud Foundry GrootFS: A daemonless container image manager
GrootFS is the new container image plugin for Garden, Cloud Foundry's
container engine. It doesn't require a daemon process, as most other engines,
and can be run as an unprivileged user, improving CF's security posture.
Categories:
Cloud Foundry
Containers
Docker
Community
Open Source
Linux
Garden
-
Don't mix goroutines and namespaces: Part 1
Hands on with Linux namespaces and threads
Categories:
containers
linux
-
Headless UI Testing with Go, Agouti, and Chrome
Acceptance tests for your UI are an excellent way to cover user functionality. Let's see how to write a simple
acceptance test in Go with Agouti and have it run headlessly in a CI environment with Chrome.
Categories:
Testing
Go
Agouti
Chrome
UI Testing
UI
-
Leveraging NSX's Features with BOSH's vSphere CPI
BOSH, a VM (Virtual Machine) orchestrator, includes the ability to
interoperate with NSX, a network virtualization platform, when deploying to a
vSphere IaaS (Infrastructure as a Service). This blog post describes deploying
VMs as the backend of an NSX Load Balancer.
Categories:
BOSH
NSX
vSphere
-
Profiling Query Compilation Time with GPORCA
GPORCA is Pivotal’s Query Optimizer for Greenplum Database and Apache HAWQ (incubating). In this post, we describe how users can profile query compilation with GPORCA. This will aid users in understanding which of GPORCA's steps is the most resource intensive, and what transformations are being triggered. Based on this information, users can provide query hints to reduce or increase the search space, see where the time and memory is being spent, and learn how to influence its decision making.
Categories:
Databases
GPDB
HAWQ
PQO
GPORCA
Query Optimization
SQL
-
Everything and the Spring Cloud Data Flow Sink
Get started with Spring Cloud Data Flow Streams by creating a custom Sink app and deploying to Pivotal Cloud Foundry.
Categories:
Spring Cloud Data Flow
Spring Cloud Data Flow Streams
-
How to Customize a BOSH Stemcell
BOSH Stemcells are Linux-based bootable disk images upon which BOSH applications
may be deployed. This blog post describes a process to customize a
stemcell (most often used to troubleshoot stemcell boot problems).
Categories:
BOSH
-
Updating a BOSH Release
Authors of a BOSH Release may want to release a new version when the
upstream application is updated. This blog post describes the process
of updating a BOSH Release while avoiding common pitfalls.
Categories:
BOSH
-
Test-Driven Development for Data Science
Unravelling Test-Driven Development for Data Science.
Categories:
TDD
Data Science
Machine Learning
Agile
Pair Programming
-
How to Set up a Distributed Elixir Cluster on Amazon EC2
Learn how to set up an Elixir cluster and how to deploy a Phoenix application on Amazon EC2. The techniques outlined in this article can equally apply to other providers such as Digital Ocean and Linode.
Categories:
Elixir
Distributed
Cluster
Phoenix
Edeliver
Deployment
Erlang
-
Improving Query Execution Speed via Code Generation
A code generation based solution inside the GPDB execution engine.
Categories:
GREENPLUM DATABASE
QUERY EXECUTION
CODE GENERATION
-
Concourse has Badges
Use Concourse's badges to display the health of your project
Categories:
Concourse
-
Concourse without a Load Balancer
nginx is a less-expensive alternative to a load balancer for a BOSH-deployed Concourse server's SSL termination.
Categories:
BOSH
Concourse
-
GPDB merge with PostgreSQL 8.3
Greenplum merge with PostgreSQL 8.3
Categories:
PostgreSQL
Greenplum Database
Databases
-
Operationalizing Data Science Models on the Pivotal Stack
Categories:
Data Science
Greenplum
SCDF
PCF
GemFire
-
Writing an Ionic 2 Application for Production
On a recent Pivotal Labs engagement, we built a hybrid application using Ionic 2. This post provides some technical considerations for building your own.
Categories:
Mobile
Hybrid Apps
Ionic
-
Faster Pipelines With Compiled BOSH Releases
Compile once, deploy many times.
Categories:
BOSH
Concourse
Testing
-
Improving Constraints In ORCA
ORCA is Pivotal's Query Optimizer for big data. We look at how we improved ORCA's understanding of logical constraints.
Categories:
Databases
Query Optimization
SQL
-
Creating a Custom Buildpack
This article will describe how to create a custom buildpack using Rust as an example language.
Categories:
Cloud Foundry
Buildpacks
Rust
CF Runtime
-
API First for Data Science
How API first can help to create smart data-driven apps.
Categories:
Data Science
Machine Learning
API First
Cloud Foundry
Smart Apps
-
Building a Native Navigation Menu for iOS with Turbolinks 5
Using Turbolinks 5 to hide your site's HTML navigation and present a native navigation view to mobile app users.
Categories:
Rails
Turbolinks
iOS
-
Using Action Cable With Cloud Foundry
A guide to configuring and deploying a Rails 5 Action Cable app to Cloud Foundry.
Categories:
Ruby On Rails
Cloud Foundry
-
Why you should stub, not shallow render, child components when testing React
A better way to avoid brittle unit tests in React
Categories:
TDD
React
Javascript
-
Data Driven Testing with Spek
Finding a usable approach to data-driven testing with Spek
Categories:
Kotlin
TDD
Spek
-
Virtual memory settings in Linux - The Problem with Overcommit
How to tune the Memory Overcommit settings in Linux
Categories:
Linux
Greenplum Database
Virtual Memory
Overcommit
-
Running GPDB using Docker
A look into how the GPDB R&D team uses Docker to increase development consistency.
Categories:
GPDB
Docker
-
SQL Stored Procedure Versioning Strategy
A versioning strategy for SQL stored procedures provides flexibility for developers both on the DB and the application side.
Categories:
SQL
Version control
-
SQL Test Driven Development with Oracle RDBMS
Test-driving SQL stored procedures using Oracle SQL Developer IDE.
Categories:
SQL
TDD
Oracle
-
Java Deserialization, JMX and CVE-2016-3427
If you use remote JMX, you need to update your JVM to address CVE-2016-3427
Categories:
Security
Java
Apache Tomcat
Pivotal tc Server
-
ByteA versus TEXT in PostgreSQL (for textual data)
One of our customers switched from MongoDB to PostgreSQL, and the migration tool created all data fields as ByteA instead of TEXT. Makes one wonder, if there is a performance difference and if TEXT could be a wiser choice.
Categories:
PostgreSQL
Performance
-
How to Deploy a Multi-homed BOSH Director to a vSphere Environment
We explore deploying a multi-homed BOSH Director to a vSphere environment to segregate networks
in a secure manner.
Categories:
BOSH
-
TDDing React + Redux
Helpful patterns for unit testing a React-Redux app
Categories:
TDD
React
Redux
Javascript
-
Using Postgres to analyze ride data
Postgres provides some fantastic functionality to help out with basic data analysis. This article will show you how to generate leaderboards and find streaks in raw sql data.
Categories:
PostgreSQL
SQL
Databases
-
Distributed Pair Programming: What Works!
Tales of pair programming on a distributed team.
Categories:
Agile
Pair Programming
-
Implementing Containers on Windows: A Deep Dive
Technical details for how we implemented containers on Windows for Pivotal
Cloud Foundry.
Categories:
CF Runtime
Diego
Windows
Containers
.NET
-
Testing JavaScript's native Promises
A straightforward look at how to apply test-driven development to native JavaScript promises.
Categories:
TDD
JavaScript
Promises
Testing
-
Faking OAuth2 Single Sign-on in Spring, Two Ways
When your Java Spring web application depends on a third-party OAuth2 single sign-on service,
tests can be slow, brittle, or difficult to control. I'll describe two ways to address these
issues by faking OAuth2 single sign-on in your tests.
Categories:
Spring
Java
Testing
-
Running Tests in AWS Lambda
Quickly and easily run your tests on AWS without the hassle of starting new
EC2 instances.
Categories:
AWS Lambda
Testing
-
ETL Journey from Oracle to Postgres
How we transferred a legacy Oracle database to a new Postgres database in a 3 hour window.
Categories:
ETL
oracle
postgres
database
-
SERIAL Datatype Performance in Greenplum Database
How to improve the performance of the SERIAL datatype in Greenplum Database
Categories:
PostgreSQL
Greenplum Database
Databases
Datatypes
SERIAL
-
"Some Blog Post" or, How I Learned to Stop Worrying and Like Red Junit Tests
Tips and tricks for writing tests that fail well. What to mock, what to name your tests, and how to
when
.
Categories:
Java
Testing
-
Building machine learning models at scale for data parallel problems on Pivotal's MPP databases
Building machine learning models (ex: scikit-learn) at scale for data parallel problems on Pivotal's MPP databases (Greenplum/HAWQ).
Categories:
Data Science
Greenplum
Procedural Languages
Python
-
Making A Useful C++ Buildpack
A useful C++ buildpack needs to consider header files and libraries, not just
make
. Here's a story about how I made a useful buildpack for a C++ web framework.
Categories:
CF Runtime
-
Algebraic Data Types In Kotlin
Getting feedback quickly about mistakes in your code is a key tenet of agile development. This article will show you how to use algebraic data types and the Kotlin compiler to get fast feedback when you have missed handling an outcome for a business use case.
Categories:
Kotlin
functional programming
-
The Journey of a Spring Boot application from Java 8 to Kotlin, part 3: Data Classes
Kotlin data classes reduce a lot of boilerplate code when it comes to writing POJOs that are used for data exchange.
Categories:
Spring Boot
Kotlin
Java
-
Current TransactionID in Greenplum Database
How to find out the current TransactionID in Greenplum Database
Categories:
PostgreSQL
Greenplum Database
Databases
-
The Journey of a Spring Boot application from Java 8 to Kotlin, part 2: Configuration Classes
What do Spring Boot configuration classes look like in Kotlin?
Categories:
Spring Boot
Kotlin
-
PgConf.Russia 2016
PgConf.Russia 2016 – Talk: How we made Greenplum Open Source
Categories:
PostgreSQL
Conference
-
Exploring at Pivotal
A candid insight into the adoption of the exploratory testing practice at Pivotal Labs.
Categories:
Agile
Exploratory Testing
Charter
CF Runtime
-
The Journey of a Spring Boot application from Java 8 to Kotlin: The Application Class
The first steps along the path of converting a fully functional Java 8/Spring Boot/Spring Cloud application to Kotlin.
Categories:
Spring Boot
Kotlin
-
Capturing Network Traffic With Docker Containers
How to capture and log internet traffic from programs using Docker containers.
Categories:
Containers
Docker
Network Traffic Monitoring
Logging & Metrics
-
PostgreSQL Meetup in Berlin, 2016-01-26
Pivotal hosted a PostgreSQL Meetup in Berlin. Speakers: Andres Freund and Oleksandr Shulgin.
Categories:
PostgreSQL
Meetup
-
Pivotal Data Open Source in 2016: community, community, community!
When it comes to Open Source, Pivotal had one kick ass of a year in 2015. Here's a sneak peak for 2016.
Categories:
Open Source
Community
Big Data
Databases
-
GPORCA, A Modular Query Optimizer, Is Now Open-Source
GPORCA has achieved an overall 5X performance improvement across all 99 industry standard benchmark queries. Now we call on the community to help take the project to the next level.
Categories:
Big Data
Databases
Query Optimization
SQL
-
Pairing for Data Scientists
Lets see how pair programming fits in the data science world.
Categories:
Data Science
Pair Programming
Agile
-
Concourse Web Logging
You need to debug your Concourse ATC server. How do you turn up the logging level to allow that?
Categories:
Concourse
-
Deploying your first .NET app on Cloud Foundry
PCF 1.6 brings with it support for .NET. Here's how to get started.
Categories:
.NET
CF Runtime
-
Intro to the Patch Command
Quick intro on how to use the patch command to edit, and revert, the text of multiple files.
Categories:
Patch
Golang
-
Abstraction, or, The Gift of Our Weak Brains
Our brains are naturally limited. This can be a curse, or it can be a gift, depending on how you look at it.
Categories:
Agile
Humans
-
Setting up Kotlin with Android and tests
First impressions of Kotlin
Categories:
Kotlin
Android
-
HTTP Trailers
Signaling failure during an HTTP stream
Categories:
Golang
HTTP
MySQL
-
The World's Smallest Concourse CI Server
How to deploy a publicly-accessible, extremely lean Concourse CI server.
Categories:
BOSH
Concourse
-
Using the Cloud Foundry Firehose Plugin
Get your Cloud Foundry Firehose logs and metrics straight to your fingertips.
Categories:
Firehose
Loggregator
CF CLI
Logging & Metrics
-
Agile and Program Logic
On some of the differences and similarities in perspective between Agile/TDD
programmers and developers of program-logic tools.
Categories:
Agile
-
Scaling up to 2000 vms with BOSH
In order to know if we can deploy 2000 vms with BOSH, we did a scaling test and this blog post list how we did it and the caveats we found.
Categories:
BOSH
-
A Team Sport
Welcome to our new Engineering Journal!
-
Hanging by a Thread
It is late on a Friday afternoon, and your web application has stopped responding to requests. The server is still
reachable, and the Apache Tomcat process is still running–there are no errors in the logs. You want to go home but
you can’t until it is fixed. What do you do?
Categories:
Tomcat
Troubleshooting
Migrated Content
-
Integrating Jenkins and Apache Tomcat for Continuous Deployment
The practice of automated continuous deployment ensures that the latest checked in code is deployed, running, and accessible to various roles within an organization. You can start practicing continuous deployment very quickly using Tomcat or tc server, Jenkins, and your source control system of choice.
Categories:
Tomcat
Continuous Integration
Migrated Content
-
JVM Tuning for Apache Tomcat
Performance Tuning the JVM for Running Apache Tomcat
Categories:
Tomcat
Performance
Migrated Content
-
Apache Tomcat GC Measurement
Setting Up Measurement of Garbage Collection in Apache Tomcat
Categories:
Tomcat
Performance
Migrated Content
-
Session Fixation Protection
An overview of session fixation attacks and how they are prevented in Apache Tomcat.
Categories:
Tomcat
Security
Migrated Context
-
Apache Tomcat jdbc-pool
Configuration and use of Apache Tomcat’s high concurrency database connection pool
Categories:
Tomcat
Database
JDBC Pool
Migrated Content