9.4 Status • The first official release. – 9.4 released on December 18th, 2014. • The latest stable release – 9.4.4 released on June 12th, 2015.
9.5 Status • Current Status – 9.5 Alpha 2 released on August 5th, 2015 • The first Beta (or another Alpha) will be coming in September. • The final release will be coming on late this year. http://www.postgresql.org/about/news/1604/
NoSQL - JSONB • “Binary JSON” – Different from JSON, a text representation – Faster for searching • With JSONB... – No duplicated keys allowed. Last wins. – Key order not preserved. – Can take advantages of GIN Index.
NoSQL - JSONB Operator Description 9.4 -> Get an element by key as a JSON object ->> Get an element by key as a text object #> Get an element by path as a JSON object #>> Get an element by path as a text object <@, @> Evaluate whether a JSON object contains a key/value pair ? Evaluate whether a JSON object contains a key or a value ?| Evaluate whether a JSON object contains ANY of keys or values ?& Evaluate whether a JSON object contains ALL of keys or values 9.5 || Insert or Update an element to a JSON object - Delete an element by key from a JSON object #- Delete an element by path from a JSON object http://www.postgresql.org/docs/9.5/static/functions-json.html
Analytics - CUBE • Calculates for all combinations of the specified columns
Analytics – GROUPING SETS • Runs multiple GROUP BY queries at once Two GROUP BYs at once.
Analytics – Materialized Views • REFRESH MATERIALIZED VIEW CONCURRENTLY myview • Refreshing a MV concurrently (in background) without exclusive lock. • Usability and availability improved.
Analytics - BRIN Index • Block Range INdex (New in 9.5) – Holds "summary“ data, instead of raw data. – Reduces index size tremendously. – Also reduces creation/maintenance cost. – Needs extra tuple fetch to get the exact record. Index Creation Index Size Select 1 record 300,000 300,000 18 16 250,000 250,000 14 s) s) 200,000 200,000 12 (m (m Blocks me 10 me 150,000 ti of 150,000 er ti 8 sed sed p 100,000 mb 100,000 p 6 Ela Nu Ela 4 50,000 50,000 2 0 0 0 Btree BRIN Btree BRIN Btree BRIN https://gist.github.com/snaga/82173bd49749ccf0fa6c
Analytics - BRIN Index • Structure of BRIN Index Holds min/max values for “Block Ranges”, Block Range 1 (128 Blocks) 128 blocks each (by default). Block Range 2 (in case a date column) Block Min. Value Max. Value Block Range 3 Range 1 1992-01-02 1992-01-28 2 1992-01-27 1992-02-08 3 1992-02-08 1992-02-16 … … … Table File
Analytics - TABLESAMPLE • Allows user to specify random BERNOULLI sampling or block level SYSTEM sampling – Would improve SELECT query performance • Need to specify – Sampling method (SYSTEM | BERNOULLI) – Fraction of the table (in a percentage) • Limitation – Currently accepts only on regular tables and materialized views http://www.postgresql.org/docs/9.5/static/sql-select.html
Analytics - TABLESAMPLE • Calculating the average of total price. – With/without TABLESAMPLE
Analytics - TABLESAMPLE Without TABLESAMPLE Cost: 44076 With SYSTEM Sampl. Cost: 1199 With BERNOULLI Sampl. Cost: 25513
INSERT, or UPDATE? • “duplicate key violation” is one of the bothersome things in database programming
INSERT … ON CONFLICT … • Now, you can INSERT or UPDATE in one statement with “ON CONFLICT”.
INSERT … ON CONFLICT … • This query updates n_comment column when INSERT conflicts on n_nationkey column. INSERT INTO nation VALUES ( 12, ‐‐ n_nationkey 'JAPAN', ‐‐ n_name 2, ‐‐ n_regionkey 'Japan (Japanese: 日本 … in East Asia.' ‐‐ n_comment ) ON CONFLICT (n_nationkey) DO UPDATE SET n_comment = EXCLUDED.n_comment; http://www.postgresql.org/docs/9.5/static/sql-insert.html
Row Level Security
Row Level Security • Row Level Security (RLS) – Allows users to define access policy to determine which rows in the table should be returned. – Disabled by default. – CREATE POLICY, ALTER POLICY, DROP POLICY • Limitation – Not applicable to the system catalog http://www.postgresql.org/docs/9.5/static/ddl-rowsecurity.html
Row Level Security • Multiple rows (user records) in the table. Define a policy to filter out with user name
Row Level Security • Each user can see only own record. “user01” can see only “user01” record “user02” can see only “user02” record
Replication and Beyond (Logical Decoding)
Replication and Beyond – Logical Decoding • “Logical” representation from replication stream – INSERT/UPDATE/DELETE operations – Can be replayed on different version/platform • pg_recvlogical command – Shows how it works • Replication can be more flexible – BDR (Bi-Directional Rep.), Slony, and more ... – Continuous Backup as well
Administration (ALTER SYSTEM)
Administration - ALTER SYSTEM • ALTER SYSTEM SET – puts new value in postgresql.auto.conf – pg_reload_conf() reloads them. – postgresql.auto.conf takes priority over postgresql.conf. • ALTER SYSTEM RESET – Remove values from postgresql.auto.conf.
Infrastructure (For Parallelization)
Dynamic Background Workers • In 9.3, background workers must start at the postmaster (listener process) startup. • After 9.4, they can be launched “on-demand” basis. • From parallelization point of view... – It allows to launch multiple background processes to execute child queries in parallel.
Dynamic Shared Memory • Shared memory can be allocated “on-demand” basis – Cf.) by background workers • Main segment (ex. shared_buffers) still fixed at startup • Also supports lightweight message queue • From parallelization point of view... – It allows to share data and communicate with several bgworker processes.
There are many other enhancements, so please try it asap.
Commitfest 2015-7~ commitfest.postgresql.org CommitFest is a process to review, fix and commit the submitted patches. • Parallel Seq scan • Waits monitoring • Support multiple synchronous standby servers • and others.. Still work in progress...
Wrap-up • One of the most developer-friendly RDBMSes in the world. • Analytics features and the performance are improving. • Things are going to parallel.
Postgres Toolkit • A script collection to manage PostgreSQL – Helps DBA to perform complicated tasks – Consists of 13 scripts as of v0.2.2. • A "Victorinox" for PostgreSQL DBA • uptime.jp/go/pt
SQL Firewall for Postgres SQL Injection prevented!
pgDay Asia 2016 • As A Joint Event with FOSSASIA 2016 – 1 Day, 2 Tracks (not fixed yet) • FOSSASIA 2016 – March 18h-20th in Singapore • Still “Work In Progress”, but mark your calendar NOW! Photo by Michael Cannon https://flic.kr/p/rieAXe