November 02, 2015

Building dynamic queries to extract regular data or Json datatype from postgreSQL

As always a lazy programmer, today my task is to get the update from a set of tables on postgres while excluding an specific pattern plus the scripts is run twice per day so I need to capture changes in a 12 hour interval.
MMMMM>..... How i can do this and get it done quickly :-)
.........agony, agony, agony.. I don't want to spend all day doing this....So, here is the solution to build the scripts quick and dirty:
select 
  'Do $$ declare today timestamp = current_timestamp - interval''12 hours''; begin Create Temp table temp_'||table_name||' on commit drop as SELECT '||string_agg(column_name,',')||' FROM '||table_schema||'.'||table_name||' where modified_date >= today; end $$; SELECT '||string_agg(column_name,',')||' FROM temp_'||table_name||';'
from 
 information_schema.columns 
where 
table_schema = 'tableSchemaHere'
and table_name !~* 'StringPartternHere'
and is_updatable = 'YES'

group by table_schema,table_name;

Ah... More agony now I need to extract data from the audit tables that uses the json datatype...
mmm.... what to do... just change the above script just a little to get data from json
select Table_Name,
'Do $$ declare today timestamp = current_timestamp - interval''12 hours'';begin  Create Temp table temp_'||table_name||' on commit drop as SELECT action,'||cast (string_agg('new_object_data->'||replace (initcap(replace(''''||column_name||'''','_',' ')),' ','')||
' AS '||replace(initcap(replace(column_name,'_',' ')),' ',''),',')as varchar(2000))||
' FROM audit.audit_history Where (action = '||'''Update'') AND object_type = '''||replace(initcap(replace(Table_Name,'_',' ')),' ','')||''' 
AND (Created_date <= today) UNION ALL SELECT action,'||cast (string_agg('original_object_data->'||replace (initcap(replace(''''||column_name||'''','_',' ')),' ','')||
' AS '||replace(initcap(replace(column_name,'_',' ')),' ',''),',')as varchar(2000))||
' FROM audit.audit_history Where (action = '||'''Delete'') AND object_type = '''||replace(initcap(replace(Table_Name,'_',' ')),' ','')||''' AND (Created_date <= today);END $$; Select * from temp_'||Replace(Table_Name,'_','')
from 
 information_schema.columns
where 
   table_schema = 'schema_here'
   
group by table_name
order by 1


quick and dirty just the way I like... Cheers.


No comments:

Post a Comment

Contact Form

Name

Email *

Message *