!wY"      !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~      !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~SafeImplements the relational Table concept. Defines all necessary data types like RTable and RTuple as well as basic relational algebra operations on RTables.(c) Nikos Karagiannidis, 2018 BSD3 nkarag@gmail.com stable POSIX None"#27Md DBFunctorMThis exception means that we have tried an Upsert operation where the source r does not have a unique set of Rtuples if grouped by the columns used in the matching condition. This simply means that we cannot determine which of the dublicate qs in the source r will overwrite the target r+, when the matching condition is satisfied. DBFunctor Error message DBFunctorIThis exception means that we have tried to do some operation between two RTablesF, which requires that the structure of the two is the same. e.g., an  Insert Into  TAB RTuples, or a UNION= or toher set operations. By "structure", we mean that the oQs and the corresponding data types must match. Essentially what we record in the C must be the same for the two rs DBFunctor3Error message indicating the operation that failed. DBFunctor#Length mismatch between the format  and the input  data RTimestampFormatLengthMismatch = RTimestampFormatLengthMismatch String String deriving(Eq,Show) instance Exception RTimestampFormatLengthMismatchOne (or both) of the input s to function  are empty DBFunctorgThis exception is thrown whenever we provide a Timestamp format with not even one valid format pattern  DBFunctorLThis exception is thrown whenever we try to access a specific column (i.e., o) of an q! and the column does not exist.  DBFunctorFormat specifier of  style DBFunctor+A map of ColumnName to Format Specification DBFunctor:Basic data type for defining the desired formatting of an q when printing an RTable (see ). DBFunctorEFor defining the column ordering (i.e., the SELECT clause in SQL)  DBFunctor*For defining the formating per Column in " style" DBFunctorRTabResult is the result of an RTable operation and is a Writer Monad, that includes the new RTable, as well as the number of RTuples returned by the operation. DBFunctor1Number of RTuples returned by an RTable operation DBFunctor_Aggregation Function type. An aggregation function receives as input a source column (i.e., a o) of a source rm and returns an aggregated value, which is the result of the aggregation on the values of the source column. DBFunctorRThis data type represents all possible aggregate operations over an RTable. Examples are : Sum, Count, Average, Min, Max but it can be any other "aggregation". The essential property of an aggregate operation is that it acts on an RTable (or on a group of RTuples - in the case of the RGroupBy operation) and produces a single RTuple.rAn aggregate operation is applied on a specific column (source column) and the aggregated result will be stored in the target column. It is important to understand that the produced aggregated RTuple is different from the input RTuples. It is a totally new RTuple, that will consist of the aggregated column(s) (and the grouping columns in the case of an RGroupBy). DBFunctor Source column DBFunctor Target column DBFunctor?here we define the aggegate function to be applied on an RTable DBFunctor9The Group By Predicate It defines the condition for two q#s to be included in the same group. DBFunctor*The Upsert Predicate. It defines when two qs should be paired in a merge operation. The matching predicate must be applied on a specific set of matching columns. The source r6 in the Upsert operation must return a unique set of qHs, if grouped by this set of matching columns. Otherwise an exception ( ) is thrown." DBFunctor(The Join Predicate. It defines when two qs should be paired.# DBFunctor$A generic binary operation on RTable$ DBFunctor%A generic unary operation on a RTable% DBFunctorTA sum type to help the specification of a column ordering (Ascending, or Descending)( DBFunctor\Definition of Relational Algebra operations. These are the valid operations between RTables* DBFunctorUnion + DBFunctor Intersection, DBFunctor Difference- DBFunctor Projection. DBFunctorFilter operation (an B' can be any function of the signature  RTuple -> Bool z so it is much more powerful than a typical SQL filter expression, which is a boolean expression of comparison operators)/ DBFunctor\Inner Join (any type of join predicate allowed. Any function with a signature of the form:  RTuple -> RTuple -> Bool < is a valid join predicate. I.e., a function which returns  when two RTuples must be paired)0 DBFunctorLeft Outer Join 1 DBFunctorRight Outer Join2 DBFunctor Semi-Join3 DBFunctor Anti-Join 4 DBFunctorRPerforms aggregation operations on specific columns and returns a singleton RTable5 DBFunctor/A Group By operation The SQL equivalent is: < SELECT colGrByList, aggList FROM... GROUP BY colGrByList \ Note that compared to SQL, we can have a more generic grouping predicate (i.e., when two qhs should belong in the same group) than just the equality of values on the common columns between two qs. Also note, that in the case of an aggregation without grouping (equivalent to a single-group group by), then the grouping predicate should be:  _ _ -> True 6 DBFunctorA combination of unary ( s e.g., / (p plist).(f pred) (i.e., RPrj . RFilter)  , in the form of an  RTable -> RTable function. k In this sense we can also include a binary operation (e.g. join), if we partially apply the join to one r, e.g., '(ij jpred rtab) . (p plist) . (f pred) 7 DBFunctorA generic binary (.8 DBFunctor Order the q s of the r{ acocrding to the specified list of Columns. First column in the input list has the highest priority in the sorting order.< DBFunctorlist of aggregates = DBFunctorthe grouping predicate> DBFunctorthe Group By list of columnsB DBFunctorFA Predicate. It defines an arbitrary condition over the columns of an q%. It is used primarily in the filter .+ operation and used in the filter function .C DBFunctor(Basic metadata for a column of an RTupleH DBFunctorBasic Metadata of an q . The q! metadata are accessed through a  o C* structure. I.e., for each column of the q, we access the CD structure to get Column-level metadata. This access is achieved by oZ. However, in order to provide the "impression" of a fixed column order per tuple (see q! definition), we provide another  , the  G o8. So in the follwoing example, if we want to access the Hh tupmdata ColumnInfo by column order, (assuming that we have N columns) we have to do the following:  (snd tupmdata)!((fst tupmdata)!0) (snd tupmdata)!((fst tupmdata)!1) ... (snd tupmdata)!((fst tupmdata)!(N-1)) 7In the same manner in order to access the column of an q2 (e.g., tup) by column order, we do the following: a tup!((fst tupmdata)!0) tup!((fst tupmdata)!1) ... tup!((fst tupmdata)!(N-1)) I DBFunctorMetadata for an RTableK DBFunctor Name of the rL DBFunctor8Tuple-level metadata other metadataM DBFunctor Primary KeyN DBFunctorIList of unique keys i.e., each sublist is a unique key column combinationO DBFunctormBasic data type to represent time. This is a strict data type, meaning whenever we evaluate a value of type O<, there must be also evaluated all the fields it contains.Z DBFunctor[Definition of the Relational Data Type. This is the data type of the values stored in each rL. This is a strict data type, meaning whenever we evaluate a value of type Z<, there must be also evaluated all the fields it contains.d DBFunctore.g., "DD/MM/YYYY"g DBFunctor-This is used only for metadata purposes (see C). The actual data type of a value is an RDataType The Text component of Date and Timestamp data constructors is the date format e.g., "DD/MM/YYYY", "DD/MM/YYYY HH24:MI:SS"n DBFunctorDefinition of the Table Nameo DBFunctorDefinition of the Column Namep DBFunctorDefinition of the Name typeq DBFunctor*Definition of the Relational Tuple. An q is implemented as a  of (o, Zi) pairs. This ensures fast access of the column value by column name. Note that this implies that the qR CANNOT have more than one columns with the same name (i.e. hashmap key) and more importantly that it DOES NOT have a fixed order of columns, as it is usual in RDBMS implementations. This gives us the freedom to perform column change operations very fast. The only place were we need fixed column order is when we try to load an r from a fixed-column structure such as a CSV file. For this reason, we have embedded the notion of a fixed column-order in the q metadata. See H.r DBFunctor0Definition of the Relational Table entity An r is a "container" of qs.s DBFunctor@Basic class to represent a data type that can be turned into an r/. It implements the concept of "tabular data" v DBFunctor Turns an r to a list of qsw DBFunctor(Creates an RTable from a list of RTuplesx DBFunctorTurns an RTuple to a Listy DBFunctorCreate an RTuple from a listz DBFunctorUse this function to compare an RDataType with the Null value because due to Null logic x == Null or x /= Null, will always return False. It returns True if input value is Null{ DBFunctorUse this function to compare an RDataType with the Null value because deu to Null logic x == Null or x /= Null, will always return False. It returns True if input value is Not Null| DBFunctorStandard date format} DBFunctor!Get the Column Names of an RTable~ DBFunctor%Returns the Column Names of an RTuple DBFunctor'Take a column value and return its type DBFunctor#Get the first RTuple from an RTable DBFunctor}Returns the value of an RTuple column based on the ColumnName key if the column name is not found, then it returns Nothing DBFunctorReturns the value of an RTuple column based on the ColumnName key if the column name is not found, then it returns a default value DBFunctorgetRTupColValue :: Returns the value of an RTuple column based on the ColumnName key if the column name is not found, then it returns Null. !!!Note that this might be confusing since there might be an existing column name with a Null value!!! DBFunctor?Operator for getting a column value from an RTuple Throws a  8 exception, if this map contains no mapping for the key. DBFunctorsSafe Operator for getting a column value from an RTuple if the column name is not found, then it returns Nothing DBFunctorMReturns the 1st parameter if this is not Null, otherwise it returns the 2nd.  DBFunctorReturns the value of a specific column (specified by name) if this is not Null. If this value is Null, then it returns the 2nd parameter. If you pass an empty RTuple, then it returns Null. Throws a  8 exception, if this map contains no mapping for the key. DBFunctorIt receives an RTuple and lookups the value at a specfic column name. Then it compares this value with the specified search value. If it is equal to the search value then it returns the specified Return Value. If not, then it returns the specified default Value, if the ignore indicator is not set, otherwise (if the ignore indicator is set) it returns the existing value. If you pass an empty RTuple, then it returns Null. Throws a  8 exception, if this map contains no mapping for the key. DBFunctorIt receives an RTuple and a default value. It returns a new RTuple which is identical to the source one but every Null value in the specified colummn has been replaced by a default value DBFunctor%It receives an RTable and a default value. It returns a new RTable which is identical to the source one but for each RTuple, for the specified column every Null value in every RTuple has been replaced by a default value If you pass an empty RTable, then it returns an empty RTable Throws a  ( exception, if the column does not exist DBFunctorIt receives an RTable, a search value and a default value. It returns a new RTable which is identical to the source one but for each RTuple, for the specified column: if the search value was found then the specified Return Value is returned else the default value is returned (if the ignore indicator is not set), otherwise (if the ignore indicator is set), it returns the existing value for the column for each qK. If you pass an empty RTable, then it returns an empty RTable Throws a  ( exception, if the column does not exist DBFunctorstripRText : O(n) Remove leading and trailing white space from a string. If the input RDataType is not an RText, then Null is returned DBFunctorConcatenates two Text  RDataTypes, in all other cases of Z it returns `. DBFunctor`Helper function to remove a character around (from both beginning and end) of an (RText t) value DBFunctor Returns an O from an input  and a format .Valid format patterns are: For year: YYYY, e.g., "0001", "2018" For month: MM, e.g., "01", "1", "12" For day: DD , e.g., "01", "1", "31" For hours: HH, HH24 e.g., "00", "23"+ I.e., hours must be specified in 24 format For minutes: MI, e.g., "01", "1", "59" For seconds: SS, e.g., "01", "1", "59"'Example of a typical format string is: "DD/MM/YYYY HH:MI:SS,If no valid format pattern is found then an  exception is thrown DBFunctor Convert an O" value to a Universal Time value () DBFunctor Convert a Universal Time value () to an O value DBFunctor7Search for the first occurence of a substring within a , and return the 1st character position, or  if the substring is not found. DBFunctor7Search for the first occurence of a substring within a 3 string and return the 1st character position, or  if the substring is not found. DBFunctor7Search for the first occurence of a substring within a \3 string and return the 1st character position, or 2 if the substring is not found, or if an non-text Z, is given as input. DBFunctormCreates an RTimestamp data type from an input timestamp format string and a timestamp value represented as a . Valid format patterns are: For year: YYYY, e.g., "0001", "2018" For month: MM, e.g., "01", "1", "12" For day: DD , e.g., "01", "1", "31" For hours: HH, HH24 e.g., "00", "23"+ I.e., hours must be specified in 24 format For minutes: MI, e.g., "01", "1", "59" For seconds: SS, e.g., "01", "1", "59"'Example of a typical format string is: "DD/MM/YYYY HH:MI:SS,If no valid format pattern is found then an  exception is thrown DBFunctor_Return the Text out of an RDataType If a non-text RDataType is given then Nothing is returned. DBFunctor Return an Z from  DBFunctorReturns  only if this is an \ DBFunctor+Standard timestamp format. For example: "DDMMYYYY HH24:MI:SS" DBFunctorQrTimeStampToText: converts an RTimestamp value to RText Valid input formats are:1.  "DD/MM/YYYY HH24:MI:SS" 2.  "YYYYMMDD-HH24.MI.SS" 3.  "YYYYMMDD" 4.  "YYYYMM" 5.  "YYYY"  DBFunctorcreateRTableMData : creates RTableMData from input given in the form of a list We assume that the column order of the input list defines the fixed column order of the RTuple.  DBFunctorcreateRTupleMdata : Creates an RTupleMData instance based on a list of (Column name, Column Data type) pairs. The order in the input list defines the fixed column order of the RTuple DBFunctoratoListColumnName: returns a list of RTuple column names, in the fixed column order of the RTuple. DBFunctor^toListColumnInfo: returns a list of RTuple columnInfo, in the fixed column order of the RTuple DBFunctoritoListRDataType: returns a list of RDataType values of an RTuple, in the fixed column order of the RTuple DBFunctorDefine equality for two C structures For two column two have "equal structure" they must have the same name and the same type. If one of the two (or both) have an h4, then they are still considered of equal structure.Creates a list of the form [(ColumnInfo, RDataType)] from a list of ColumnInfo and an RTuple. The returned list respects the order of the [ColumnInfo]. It guarantees that RDataTypes will be in the same column order as [ColumnInfo], i.e., the correct RDataType for the correct column DBFunctorcreateRDataType: Get a value of type a and return the corresponding RDataType. The input value data type must be an instance of the Typepable typeclass from Data.Typeable DBFunctor Returns an 5 with a custom aggregation function provided as input DBFunctorThe StrAgg aggregate operation This is known as "string_agg"" in Postgresql and "listagg" in Oracle. It aggregates the values of a text Z" column with a specified delimiter  DBFunctorWA helper function that implements the basic fold for the raggStrAgg aggregation  DBFunctorThe Sum aggregate operation  DBFunctorWA helper function in raggSum that implements the basic fold for sum aggregation  DBFunctor>The Count aggregate operation Count aggregation (no distinct)  DBFunctor[A helper function in raggCount that implements the basic fold for Count aggregation  DBFunctor;The CountStar aggregate operation Returns the number of q s in the r (i.e., count(*) in SQL)   DBFunctorcA helper function in raggCountStar that implements the basic fold for CountStar aggregation  DBFunctorFThe CountDist aggregate operation Count distinct aggregation (i.e., count(distinct col)@ in SQL). Returns the distinct number of values for this column. DBFunctorcA helper function in raggCountDist that implements the basic fold for CountDist aggregation  DBFunctorThe Average aggregate operation DBFunctorThe Max aggregate operation DBFunctorWA helper function in raggMax that implements the basic fold for Max aggregation  DBFunctorThe Min aggregate operation DBFunctorWA helper function in raggMin that implements the basic fold for Min aggregation  DBFunctor@ropU operator executes a unary ROperation. A short name for the  function DBFunctorExecute a Unary ROperation DBFunctorCropUres operator executes a unary ROperation. A short name for the  function DBFunctor)Execute a Unary ROperation and return an  DBFunctorAropB operator executes a binary ROperation. A short name for the  function DBFunctorExecute a Binary ROperation DBFunctorDropBres operator executes a binary ROperation. A short name for the  function DBFunctor*Execute a Binary ROperation and return an  DBFunctorTest whether an RTable is empty DBFunctorTest whether an RTuple is empty DBFunctor#emptyRTable: Create an empty RTable DBFunctorACreates an empty RTuple (i.e., one with no column,value mappings) DBFunctor&Creates an RTable with a single RTuple DBFunctorEcreateRTuple: Create an Rtuple from a list of column names and values DBFunctorCreates a Null q+ based on a list of input Column Names. A ` q is an q( where all column names correspond to a ` value (` is a data constructor of Z) DBFunctorReturns  if the input q( is a Null RTuple, otherwise it returns 8 Note that a Null RTuple has all its values equal with `; but it still has columns. This is different from an empty q, which is an q0 withi no columns and no values whatsoever. See . DBFunctorThis is a fold operation on a r that returns an r. It is similar with : 3 foldr' :: (a -> b -> b) -> b -> Vector a -> b B of Vector, which is an O(n) Right fold with a strict accumulator DBFunctorThis is a fold operation on a r that returns an Z value. It is similar with : 3 foldr' :: (a -> b -> b) -> b -> Vector a -> b B of Vector, which is an O(n) Right fold with a strict accumulator DBFunctorThis is a fold operation on r that returns an r. It is similar with : 2 foldl' :: (a -> b -> a) -> a -> Vector b -> a A of Vector, which is an O(n) Left fold with a strict accumulator DBFunctorThis is a fold operation on r that returns an Z value It is similar with : 2 foldl' :: (a -> b -> a) -> a -> Vector b -> a A of Vector, which is an O(n) Left fold with a strict accumulator DBFunctorMap function over an r. DBFunctorO(n) Transform this q& by applying a function to every value DBFunctorO(n) Transform this q& by applying a function to every value DBFunctorCreates an RTuplesRet type DBFunctor6Return the number embedded in the RTuplesRet data type DBFunctorlCreates an RTabResult (i.e., a Writer Monad) from a result RTable and the number of RTuples that it returned DBFunctor8Returns the info "stored" in the RTabResult Writer Monad DBFunctoraReturns the "log message" in the RTabResult Writer Monad, which is the number of returned RTuples DBFunctorremoveColumn : removes a column from an RTable. The column is specified by ColumnName. If this ColumnName does not exist in the RTuple of the input RTable then nothing is happened, the RTuple remains intact. DBFunctor%addColumn: adds a column to an RTable DBFunctor7Filter (i.e. selection operator). A short name for the  runRFilter function DBFunctorExecutes an RFilter operation DBFunctor1RTable Projection operator. A short name for the  function DBFunctorkImplements RTable projection operation. If a column name does not exist, then an empty RTable is returned. DBFunctorImplements RTable projection operation. If a column name does not exist, then the returned RTable includes this column with a Null value. This projection implementation allows missed hits. DBFunctorreturns the N first qs of an r DBFunctorr* anti-join operator. A short name for the  function DBFunctoroImplements the anti-Join operation between two RTables (any type of join predicate is allowed) It returns the qs from the left r that DONT match with the right r. DBFunctorr* semi-join operator. A short name for the  function DBFunctoroImplements the semi-Join operation between two RTables (any type of join predicate is allowed) It returns the qs from the left r that match with the right r. Note that if an q from the left r matches more than one qs from the right r4 the semi join operation will return only a single q.  DBFunctorr+ Inner Join Operator. A short name for the  function DBFunctorImplements an Inner Join operation between two RTables (any type of join predicate is allowed) Note that this operation is implemented as a  union, which means "the first Map (i.e., the left RTuple) will be prefered when dublicate keys encountered with different values. That is, in the context of joining two RTuples the value of the first (i.e., left) RTuple on the common key will be prefered. DBFunctor#Implements an Inner Join operation between two RTables (any type of join predicate is allowed) This Inner Join implementation follows Oracle DB's convention for common column names. When we have two tuples t1 and t2 with a common column name (lets say "Common"), then the resulting tuple after a join will be "Common", "Common_1", so a "_1" suffix is appended. The tuple from the left table by convention retains the original column name. So "Column_1" is the column from the right table. If "Column_1" already exists, then "Column_2" is used. DBFunctorJoins two RTuples into one. In this join we follow Oracle DB's convention when joining two tuples with some common column names. When we have two tuples t1 and t2 with a common column name (lets say Common2), then the resulitng tuple after a join will be Common, Common_1r, so a "_1" suffix is appended. The tuple from the left table by convention retains the original column name. So Column_1) is the column from the right table. If Column_1 already exists, then Column_2 is used. DBFunctor6RTable Left Outer Join Operator. A short name for the  function DBFunctorImplements a Left Outer Join operation between two RTables (any type of join predicate is allowed), i.e., the rows of the left RTable will be preserved. Note that when dublicate keys encountered that is, since the underlying structure for an RTuple is a Data.HashMap.Strict, only one value per key is allowed. So in the context of joining two RTuples the value of the left RTuple on the common key will be prefered.Implements a Left Outer Join operation between two RTables (any type of join predicate is allowed), i.e., the rows of the left RTable will be preserved. A Left Join : + tabLeft LEFT JOIN tabRight ON joinPred i where tabLeft is the preserving table can be defined as: the Union between the following two RTables:EThe result of the inner join: tabLeft INNER JOIN tabRight ON joinPredThe rows from the preserving table (tabLeft) that DONT satisfy the join condition, enhanced with the columns of tabRight returning Null values.rThe common columns will appear from both tables but only the left table column's will retain their original name.  DBFunctorReceives two lists of o!s and returns the unique list of os after concatenating the two and removing the names from the second one that are a prefix of the first one. This function is intended to dedublicate common columns after a join (see ij ), where ColA# for example, will also appear as ColA_1-. This function DOES NOT dedublicate columns ColA and ColAsomeSuffix, only cases like this one  ColName_Num8 (e.g., ColName_1, ColName_2, etc.) Here is an example:mgetUniqueColumnNames ["ColA","ColB"] ["ColC","ColA", "ColA_1", "ColA_2", "ColA_A", "ColA_hello", "ColAhello"]8["ColA","ColB","ColC","ColA_A","ColA_hello","ColAhello"] DBFunctor7RTable Right Outer Join Operator. A short name for the  function DBFunctorImplements a Right Outer Join operation between two RTables (any type of join predicate is allowed), i.e., the rows of the right RTable will be preserved. A Right Join : , tabLeft RIGHT JOIN tabRight ON joinPred j where tabRight is the preserving table can be defined as: the Union between the following two RTables:EThe result of the inner join: tabLeft INNER JOIN tabRight ON joinPredThe rows from the preserving table (tabRight) that DONT satisfy the join condition, enhanced with the columns of tabLeft returning Null values.sThe common columns will appear from both tables but only the right table column's will retain their original name.  DBFunctorImplements a Right Outer Join operation between two RTables (any type of join predicate is allowed) i.e., the rows of the right RTable will be preserved. Note that when dublicate keys encountered that is, since the underlying structure for an RTuple is a Data.HashMap.Strict, only one value per key is allowed. So in the context of joining two RTuples the value of the right RTuple on the common key will be prefered.6RTable Full Outer Join Operator. A short name for the  function DBFunctor:Implements a Full Outer Join operation between two RTables (any type of join predicate is allowed) A full outer join is the union of the left and right outer joins respectively. The common columns will appear from both tables but only the left table column's will retain their original name (just by convention). DBFunctor,RTable Union Operator. A short name for the  function DBFunctorAImplements the union of two RTables as a union of two lists (see  ). Duplicates, and elements of the first list, are removed from the the second list, but if the first list contains duplicates, so will the result:Implements the union of two RTables. Note that dublicate q elimination takes places. DBFunctorIImplements the union-all of two RTables. I.e., a union without dublicate q elimination. Runs in O(m+n). DBFunctor3RTable Intersection Operator. A short name for the  function DBFunctor+Implements the intersection of two RTables  DBFunctor1RTable Difference Operator. A short name for the  function DBFunctorKImplements the set Difference of two RTables as the diff of two lists (see  ). DBFunctor+Aggregation Operator. A short name for the  function DBFunctorImplements the aggregation operation on an RTable It aggregates the specific columns in each AggOperation and returns a singleton RTable i.e., an RTable with a single RTuple that includes only the agg columns and their aggregated value. DBFunctor(Order By Operator. A short name for the  function DBFunctor4Implements the ORDER BY operation. First column in the input list has the highest priority in the sorting order We treat Null as the maximum value (anything compared to Null is smaller). This way Nulls are send at the end (i.e., "Nulls Last" in SQL parlance). This is for Asc ordering. For Desc ordering, we have the opposite. Nulls go first and so anything compared to Null is greater. @ SQL example with q as (select case when level < 4 then level else NULL end c1 -- , level c2 from dual connect by level < 7 ) select * from q order by c1>C1 ---- 1 2 3 Null Null Nullwith q as (select case when level < 4 then level else NULL end c1 -- , level c2 from dual connect by level < 7 ) select * from q order by c1 desc DBFunctor(Group By Operator. A short name for the  function DBFunctor'Implement a grouping operation over an rL. No aggregation takes place. It returns the individual groups as separate r)s in a list. In total the initial set of qs is retained. If an empty r; is provided as input, then a ["empty RTable"] is returned. DBFunctorConcatenates a list of r2s to a single RTable. Essentially, it unions (see ) all rs of the list. DBFunctor'Implement a grouping operation over an r*. No aggregation takes place. The output r has exactly the same q[s, as the input, but these are grouped based on the input grouping predicate. If an empty r% is provided as input, then an empty r is returned. DBFunctor*Implements the GROUP BY operation over an r.  DBFunctor;Helper function to returned a fixed Ordering Specification % from a list of os DBFunctorA short name for the  function DBFunctorrunCombinedROp: A Higher Order function that accepts as input a combination of unary ROperations e.g., (p plist).(f pred) expressed in the form of a function (RTable -> Rtable) and applies this function to the input RTable. In this sense we can also include a binary operation (e.g. join), if we partially apply the join to one RTable e.g., (ij jpred rtab) . (p plist) . (f pred) DBFunctor@O(n) append an RTuple to an RTable Please note that this is an  immutable implementation of an rD insert. This simply means that the insert operation returns a new r# and does not affect the original r. DBFunctorAO(n) prepend an RTuple to an RTable Please note that this is an  immutable implementation of an rD insert. This simply means that the insert operation returns a new r# and does not affect the original r. DBFunctor Insert an r to an existing r. This is equivalent to an INSERT INTO SELECT+ caluse in SQL. We want to insert into an rO the results of a "subquery", which in our case is materialized via the input r. Please note that this is an  immutable implementation of an rD insert. This simply means that the insert operation returns a new r# and does not affect the original r(. Also note that the source and target rDs should have the same structure. By "structure", we mean that the oQs and the corresponding data types must match. Essentially what we record in the C must be the same for the two rs. Otherwise a  exception will be thrown. DBFunctorAUpsert (Update+Insert, aka Merge) Operation. We provide a source r and a matching condition ( ) to the qs of the target r. An q from the target r might match to a single only q in the source r;, or not match at all. If it is matched to more than one qs then an exception () is thrown. When an q from the target r is matched to a source q/, then the corresponding columns of the target q9 are updated with the new values provided in the source q". This takes place for the target q/s that match but also that satisfy the input B1. Thus we can restrict further with a filter the qs of the target r8 where the update will take place. Finally, the source q#s that did not match to the target r(, are inserted (appended) to the target rPlease note that this is an  immutable implementation of an rD upsert. This simply means that the upsert operation returns a new r# and does not affect the original r5. Moreover, if we have multiple threads updating an r?, due to immutability, each thread "sees" its own copy of the r3 and thus there is no need for locking the updated q s, as happens in a common RDBMS.%Also note that the source and target rDs should have the same structure. By "structure", we mean that the oQs and the corresponding data types must match. Essentially what we record in the C must be the same for the two rs. Otherwise a  exception will be thrown.  An Example: Source RTable: src = Id | Msg | Other ----|---------------|------- 1 | "hello2" |"a" 2 | "world2" |"a" 3 | "new" |"a" Target RTable: trg = Id | Msg | Other ----|---------------|------- 1 | "hello1" |"b" 2 | "world1" |"b" 4 | "old" |"b" 5 | "hello" |"b" >>> upsertRTab src RUpsertPredicate {matchCols = ["Id"], matchPred = \t1 t2 -> t1 <!> "Id" == t2 <!> "Id" } ["Msg"] (\t -> let msg = case toText (t <!> "Msg") of Just t -> t Nothing -> pack "" in (take 5 msg) == (pack "hello") ) -- Msg like "hello%" trg Result RTable: rslt = Id | Msg | Other ----|---------------|------- 1 | "hello2" |"b" (Note that only column "Msg" has been overwritten, as per the 3rd argument) 2 | "world1" |"b" 3 | "new" |"a" 4 | "old" |"b" 5 | "hello" |"b"  DBFunctor$Compares the structure of the input rs and returns : if these are the same. By "structure", we mean that the oQs and the corresponding data types must match. Essentially what we record in the C must be the same for the two rbs. Note that in the case of two columns having the same name but one of the two (or both) have a F equal to hB, then this function assumes that they are the same (i.e., equal Cs). DBFunctor$Compares the structure of the input qs and returns : if these are the same. By "structure", we mean that the oQs and the corresponding data types must match. Essentially what we record in the C must be the same for the two qs DBFunctorDelete q s from an r based on an B. Please note that this is an  immutable implementation of an rD update. This simply means that the delete operation returns a new r. So, the original rx remains unchanged and no deletion in-place takes place whatsoever. Moreover, if we have multiple threads deleting an r?, due to immutability, each thread "sees" its own copy of the r3 and thus there is no need for locking the deleted q s, as happens in a common RDBMS. DBFunctorUpdate an RTable. The input includes a list of (ColumnName, new Value) pairs. Also a filter predicate is specified, in order to restrict the update only to those q;s that fulfill the predicate. Please note that this is an  immutable implementation of an rD update. This simply means that the update operation returns a new r that includes all the qs of the original rW, both the ones that have been updated and the others that have not. So, the original rv remains unchanged and no update in-place takes place whatsoever. Moreover, if we have multiple threads updating an r?, due to immutability, each thread "sees" its own copy of the r3 and thus there is no need for locking the updated q s, as happens in a common RDBMS. DBFunctorMUpdate an RTuple at a specific column specified by name with a value. If the oA exists, then the value is updated with the input value. If the o does not exist, then a   exception is thrown. DBFunctorUpsert (update/insert) an RTuple at a specific column specified by name with a value If the cname key is not found then the (columnName, value) pair is inserted. If it exists then the value is updated with the input value. DBFunctor/Generates a default Column Format Specification DBFunctor'Generates a Column Format Specification DBFunctor+Generate an RTupleFormat data type instance DBFunctorGenerate a default RTupleFormat data type instance. In this case the returned column order (Select list), will be unspecified and dependant only by the underlying structure of the q () DBFunctorSafe  printRfTable alternative that returns an r, so as to give the ability to handle exceptions gracefully, during the evaluation of the input RTable. Example: do p <- (eitherPrintfRTable printfRTable myFormat myRTab) :: IO (Either SomeException ()) case p of Left exc -> putStrLn $ "There was an error in the Julius evaluation: " ++ (show exc) Right _ -> return ()  DBFunctorQprints an RTable with an RTuple format specification. It can be used instead of + when one of the following two is required:Oa) When we want to specify the order that the columns will be printed on screenCb) When we want to specify the formatting of the values by using a -like   DBFunctorSafe  alternative that returns an r, so as to give the ability to handle exceptions gracefully, during the evaluation of the input RTable. Example: do p <- (eitherPrintRTable printRTable myRTab) :: IO (Either SomeException ()) case p of Left exc -> putStrLn $ "There was an error in the Julius evaluation: " ++ (show exc) Right _ -> return ()  DBFunctor.printRTable : Print the input RTable on screen DBFunctorReturns the max length of the String representation of each value, for each column of the input RTable. It returns the lengths in the column order specified by the input RTupleFormat parameter DBFunctorhReturns the max length of the String representation of each value, for each column of the input RTable.  DBFunctoruhelper function in order to format the value of a column It will append at the end of the string n number of spaces. DBFunctoruhelper function in order to format the value of a column It will append at the end of the string n number of spaces. DBFunctorhelper function that prints a continuous line adjusted to the size of the input RTable The column order is specified by the input RTupleFormat parameter DBFunctorVhelper function that prints a continuous line adjusted to the size of the input RTable DBFunctorPrints the input RTable's header (i.e., column names) on screen. The column order is specified by the corresponding RTupleFormat parameter. DBFunctorSprintRTableHeader : Prints the input RTable's header (i.e., column names) on screen DBFunctorAPrints an RTuple on screen (only the values of the columns) [Int] is a List of width per column to be used in the box definition The column order as well as the formatting specifications are specified by the first parameter. We assume that the order in [Int] corresponds to that of the RTupleFormat parameter. DBFunctorPrints an RTuple on screen (only the values of the columns) [Int] is a List of width per column to be used in the box definition  DBFunctornTurn the value stored in a RDataType into a String in order to be able to print it wrt to the specified format  DBFunctorTurn the value stored in a RDataType into a String in order to be able to print it Values are transformed with a default formatting.  DBFunctorCIn order to be able to force full evaluation up to Normal Form (NF) DBFunctor-In order to be able to use (/) with RDataType DBFunctor|We need to explicitly specify equation of RDataType due to SQL NULL logic (i.e., anything compared to NULL returns false): R Null == _ = False, _ == Null = False, Null /= _ = False, _ /= Null = False. i IMPORTANT NOTE: Of course this means that anywhere in your code where you have something like this:  x == Null or x /= Null, a will always return False and thus it is futile to do this comparison. You have to use the is z function instead.  DBFunctorEIn order to be able to force full evaluation up to Normal Form (NF) <https://www.fpcomplete.com/blog/2017/09/all-about-strictness; DBFunctorColumnName key DBFunctor Input RTuple DBFunctor Output value DBFunctorPDefault value to return in the case the column name does not exist in the RTuple DBFunctorColumnName key DBFunctor Input RTuple DBFunctor Output value DBFunctorColumnName key DBFunctor Input RTuple DBFunctor Output value DBFunctor Input RTuple DBFunctorColumnName key DBFunctor Output value DBFunctor Input RTuple DBFunctorColumnName key DBFunctor Output value DBFunctor input value DBFunctor-default value returned if input value is Null DBFunctor output value DBFunctorColumnName key DBFunctor(value returned if original value is Null DBFunctor input RTuple DBFunctor output value DBFunctorColumnName key DBFunctor Search value DBFunctor Return value DBFunctorDefault value  DBFunctorIgnore default indicator  DBFunctor input RTuple DBFunctorColumnName key DBFunctor/Default value in the case of Null column values DBFunctorinput RTuple  DBFunctor output RTuple DBFunctorColumnName key DBFunctorDefault value  DBFunctor input RTable DBFunctorColumnName key DBFunctor Search value DBFunctor Return value DBFunctorDefault value  DBFunctorIgnore default indicator  DBFunctor input RTable DBFunctor input string DBFunctor)Format string e.g., "DD/MM/YYYY HH:MI:SS" DBFunctorTimestamp string DBFunctorsubstring to search for DBFunctorstring to be searched DBFunctor5Position within input string of substr 1st character  DBFunctorsubstring to search for DBFunctorstring to be searched DBFunctor5Position within input string of substr 1st character  DBFunctorsubstring to search for DBFunctorstring to be searched DBFunctor5Position within input string of substr 1st character  DBFunctor+Format string e.g., "DD/MM/YYYY HH24:MI:SS" DBFunctorTimestamp string DBFunctor+Output format e.g., "DD/MM/YYYY HH24:MI:SS" DBFunctorInput RTimestamp  DBFunctor Output RText DBFunctorPrimary Key. [] if no PK exists DBFunctor0list of unique keys. [] if no unique keys exists DBFunctor input value DBFunctoroutput RDataType DBFunctorcustom aggregation function  DBFunctor source column DBFunctor target column DBFunctor source column DBFunctor target column DBFunctordelimiter string  DBFunctor source column DBFunctor target column DBFunctor source column DBFunctor target column DBFunctor1target column to save the result aggregated value DBFunctor source column DBFunctor target column DBFunctor source column DBFunctor target column DBFunctor source column DBFunctor target column DBFunctor source column DBFunctor target column DBFunctorinput ROperation DBFunctor input RTable DBFunctor output RTable DBFunctorinput ROperation DBFunctor input RTable DBFunctoroutput: Result of operation DBFunctorinput ROperation DBFunctor input RTable1 DBFunctorinput RTable2  DBFunctor output RTabl DBFunctorinput ROperation DBFunctor input RTable1 DBFunctorinput RTable2  DBFunctoroutput: Result of operation DBFunctor&input list of (columnname,value) pairs DBFunctor input pair  DBFunctoroutput Writer Monad DBFunctorColumn to be removed DBFunctor input RTable  DBFunctoroutput RTable  DBFunctorname of the column to be added DBFunctorZDefault value of the new column. All RTuples will initially have this value in this column DBFunctor Input RTable DBFunctor Output RTable DBFunctor>list of column names to be included in the final result RTable DBFunctor>list of column names to be included in the final result RTable DBFunctor number of N q s to return DBFunctorinput r DBFunctoroutput r DBFunctorInput Aggregate Operations DBFunctor Input RTable DBFunctorOutput singleton RTable DBFunctorInput ordering specification DBFunctor Input RTable DBFunctor Output RTable DBFunctor3Grouping predicate, in order to form the groups of qs (it defines when two q's should be included in the same group) DBFunctorList of grouping column names (GROUP BY clause in SQL) We assume that all RTuples in the same group have the same value in these columns DBFunctorinput r DBFunctoroutput list of r's where each one corresponds to a group DBFunctor3Grouping predicate, in order to form the groups of qs (it defines when two q's should be included in the same group) DBFunctorMList of grouping column names (GROUP BY clause in SQL) We assume that all q8s in the same group have the same value in these columns DBFunctorinput r DBFunctoroutput r DBFunctor}Grouping predicate, in order to form the groups of RTuples (it defines when two RTuples should be included in the same group) DBFunctor.Aggregations to be applied on specific columns DBFunctorList of grouping column names (GROUP BY clause in SQL) We assume that all RTuples in the same group have the same value in these columns DBFunctor input RTable DBFunctor output RTable DBFunctorinput combined RTable operation DBFunctor7input RTable that the input function will be applied to DBFunctor output RTable DBFunctorSource r to be inserted DBFunctorTarget r DBFunctor Final Result DBFunctorSource r!, i.e., the equivalent to an SQL USING subclause  DBFunctorThe q+ matching predicate for the merge operation DBFunctor\List of column names to be updated with the corresponding new values coming from the source qs that match with the target qs based on the   DBFunctor#A filter that specifies the target qs to be updated DBFunctor The target r DBFunctor Final Result DBFunctorPredicate specifying the Rtuples that must be deleted DBFunctorr" that the deletion will be applied DBFunctorResult r DBFunctorDList of column names to be updated with the corresponding new values DBFunctorCAn RTuple -> Bool function that specifies the RTuples to be updated DBFunctor Input RTable DBFunctor Output RTable DBFunctor$key where the update will take place DBFunctor new value DBFunctor input RTuple DBFunctor output RTuple DBFunctor$key where the upsert will take place DBFunctor new value DBFunctor input RTuple DBFunctor output RTuple DBFunctorColumn Select list  DBFunctorColumn Format Map DBFunctorOutput DBFunctornumber of spaces to add DBFunctor input String DBFunctor output string DBFunctornumber of spaces to add DBFunctorcharacter to add DBFunctor input String DBFunctor output string DBFunctor&Specifies the appropriate column order DBFunctor;a List of width per column to be used in the box definition DBFunctor*the char with which the line will be drawn DBFunctor;a List of width per column to be used in the box definition DBFunctor*the char with which the line will be drawn DBFunctorSpecifies Column order DBFunctor;a List of width per column to be used in the box definition DBFunctor;a List of width per column to be used in the box definition   !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ`[\]^_abcdefgmihjklnopqrstuvwxyz{|}~rqZ`[\]^_abcdefOPQRSTUVIJKLMNHCDEFGpongmihjklstu()*+,-./0123456789:;<=>?@A$#B" !WXY|z{vxwy}~    %&''Implements ETL operations over RTables.(c) Nikos Karagiannidis, 2018 BSD3 nkarag@gmail.com stable POSIX Noneё!- DBFunctorETLmapping : it is the equivalent of a mapping in an ETL tool and consists of a series of ETLOperations that are applied, one-by-one, to some initial input RTable, but if binary ETLOperations are included in the ETLMapping, then there will be more than one input RTables that the ETLOperations of the ETLMapping will be applied to. When we apply (i.e., run) an ETLOperation of the ETLMapping we get a new RTable, which is then inputed to the next ETLOperation, until we finally run all ETLOperations. The purpose of the execution of an ETLMapping is to produce a single new RTable as the result of the execution of all the ETLOperations of the ETLMapping. In terms of database operations an ETLMapping is the equivalent of an CREATE AS SELECT (CTAS) operation in an RDBMS. This means that anything that can be done in the SELECT part (i.e., column projection, row filtering, grouping and join operations, etc.) in order to produce a new table, can be included in an ETLMapping.?An ETLMapping is executed with the etl (runETLmapping) operatorImplementation: An ETLMapping is implemented as a binary tree where the node represents the ETLOperation to be executed and the left branch is another ETLMapping, while the right branch is an RTable (that might be empty in the case of a Unary ETLOperation). Execution proceeds from bottom-left to top-right. This is similar in concept to a left-deep join tree. In a Left-Deep ETLOperation tree the "pipe" of ETLOperations comes from the left branches always. The leaf node is always an ETLMapping with an ETLMapEmpty in the left branch and an RTable in the right branch (the initial RTable inputed to the ETLMapping). In this way, the result of the execution of each ETLOperation (which is an RTable) is passed on to the next ETLOperation. Here is an example: { A Left-Deep ETLOperation Tree final RTable result / etlOp3 / etlOp2 rtab2 / A leaf-node --> etlOp1 emptyRTab / ETLMapEmpty rtab1 You see that always on the left branch we have an ETLMapping data type (i.e., a left-deep ETLOperation tree). So how do we implement the following case?  final RTable result / A leaf-node --> etlOp1 / rtab1 rtab2 bThe answer is that we "model" the left RTable (rtab1 in our example) as an ETLMapping of the form: S ETLMapLD { etlOp = ETLcOp{cmap = ColMapEmpty}, tabL = ETLMapEmpty, tabR = rtab1 } So we embed the rtab1 in a ETLMapping, which is a leaf (i.e., it has an empty prevMap), the rtab1 is in the right branch (tabR) and the ETLOperation is the EmptyColMapping, which returns its input RTable when executed. We can use function [ for this job. So it becomes ] A leaf-node --> etlOp1 / rtabToETLMapping rtab1 rtab2 >In this manner, a leaf-node can also be implemented like this: Z final RTable result / etlOp3 / etlOp2 rtab2 / A leaf-node --> etlOp1 emptyRTab / rtabToETLMapping rtab1 emptyRTable . DBFunctor an empty node/ DBFunctora Left-Deep node0 DBFunctora Right-Deep node1 DBFunctora Balanced node2 DBFunctor#the ETLOperation to be executed 3 DBFunctorWthe left-branch corresponding to the previous ETLOperation, which is input to this one.4 DBFunctorthe right branch corresponds to another RTable (for binary ETL operations). If this is a Unary ETLOperation then this field must be an empty RTable.5 DBFunctorthe left-branch corresponds to another RTable (for binary ETL operations). If this is a Unary ETLOperation then this field must be an empty RTable. 6 DBFunctorXthe right branch corresponding to the previous ETLOperation, which is input to this one.7 DBFunctorthe left-branch corresponding to the previous ETLOperation, which is input to this one. If this is a Unary ETLOperation then this field might be an empty ETLMapping. 8 DBFunctorthe right branch corresponding corresponding to the previous ETLOperation, which is input to this one. -- If this is a Unary ETLOperation then this field might be an empty ETLMapping.9 DBFunctor7An ETL operation applied to an RTable can be either an (C (a relational agebra operation like join, filter etc.) defined in  RTable.Core module, or an ? applied to an r> DBFunctorA Column Transformation function data type. It is used in order to define an arbitrary column-level transformation (i.e., from a list of N input Column-Values we produce a list of M derived (output) Column-Values). A Column value is represented with the Z.? DBFunctorQThis is the basic data type to define the column-to-column mapping from a source r to a target r. Essentially, an ?3 represents the column-level transformations of an q that will yield a target q. A mapping is simply a triple of the form ( Source-Column(s), Target-Column(s), Transformation, RTuple-Filter), where we define the source columns over which a transformation (i.e. a function) will be applied in order to yield the target columns. Also, an B0 (i.e. a filter) might be applied on the source q. Remember that an qK is essentially a mapping between a key (the Column Name) and a value (the Z value). So the various ?M data constructors below simply describe the possible modifications of an q! orginating from its own columns.So, we can have the following mapping types: a) single-source column to single-target column mapping (1 to 1), the source column will be removed or not based on the F4 flag (dublicate column names are not allowed in an q) b) multiple-source columns to single-target column mapping (N to 1), The N columns will be merged to the single target column based on the transformation. The N columns will be removed from the RTuple or not based on the F4 flag (dublicate column names are not allowed in an q) c) single-source column to multiple-target columns mapping (1 to M) the source column will be "expanded" to M target columns based ont he transformation. the source column will be removed or not based on the F4 flag (dublicate column names are not allowed in an q) d) multiple-source column to multiple target columns mapping (N to M) The N source columns will be mapped to M target columns based on the transformation. The N columns will be removed from the RTuple or not based on the F2 flag (dublicate column names are not allow in an q)+Some examples of mapping are the following:  ( Start_Date, No,  StartDate, t -> True) -- copy the source value to target and dont remove the source column, so the target RTuple will have both columns  Start_Date and  StartDateX -- with the exactly the same value) ([Amount, Discount], Yes,  FinalAmount, ([a, d] -> a * d) ) --  FinalAmount is a derived column based on a function applied to the two source columns. -- In the final RTuple we remove the two source columns. An ? can be applied with the S (runColMapping) operatorA DBFunctor>single-source column to single-target column mapping (1 to 1).B DBFunctor@multiple-source columns to single-target column mapping (N to 1)C DBFunctor@single-source column to multiple-target columns mapping (1 to N)D DBFunctorsmultiple-source column to multiple target columns mapping (N to M) R DBFunctorConstructs an RColMapping. This is the suggested method for creating a column mapping and not by calling the data constructors directly.S DBFunctor&runCM operator executes an RColMapping If a target-column has the same name with a source-column and a DontRemoveSrc (i.e., removeSrcCol == No) has been specified, then the (target-column, target-value) key-value pair, overwrites the corresponding (source-column, source-value) key-value pair! DBFunctorcApply an RColMapping to a source RTable and produce a new RTable. If a target-column has the same name with a source-column and a DontRemoveSrc (i.e., removeSrcCol == No) has been specified, then the (target-column, target-value) key-value pair, overwrites the corresponding (source-column, source-value) key-value pair. If a filter is embedded in the ?, then the returned r will include only the q$s that satisfy the filter predicate.T DBFunctorexecutes a Unary ETL Operation" DBFunctorexecutes an ETL OperationU DBFunctorexecutes a Binary ETL Operation# DBFunctorexecutes an ETL OperationV DBFunctor<Creates a left-deep leaf ETL Mapping, of the following form: { A Left-Deep ETLOperation Tree final RTable result / etlOp3 / etlOp2 rtab2 / A leaf-node --> etlOp1 emptyRTab / ETLMapEmpty rtab1 W DBFunctor1creates a Binary operation leaf node of the form: Y A leaf-node --> etlOp1 / rtabToETLMapping rtab1 rtab2 X DBFunctorDConnects an ETL Mapping to a left-deep ETL Mapping tree, of the form { A Left-Deep ETLOperation Tree final RTable result / etlOp3 / etlOp2 rtab2 / A leaf-node --> etlOp1 emptyRTab / ETLMapEmpty rtab1 Example:  -- connect a Unary ETL mapping (etlOp2) etlOp2 / etlOp1 emptyRTab => connectETLMapLD etlOp2 emptyRTable prevMap -- connect a Binary ETL Mapping (etlOp3) etlOp3 / etlOp2 rtab2 => connectETLMapLD etlOp3 rtab2 prevMap Note that the right branch (RTable) appears first in the list of input arguments of this function and the left branch (ETLMapping) appears second. This is strange, and one could thought that it is a mistake (i.e., the left branch should appear first and the right branch second) since we are reading from left to right. However this was a deliberate choice, so that we leave the left branch (which is the connection point with the previous ETLMapping) as the last argument, and thus we can partially apply the argumenets and get a new function with input parameter only the previous mapping. This is very helpfull in function compositionY DBFunctorThis operator executes an -$ DBFunctor Executes an -Z DBFunctorThis operator executes an - and returns the  WriterI Monad that embedds apart from the resulting RTable, also the number of q s returned[ DBFunctor Model an r as an -+ which when executed will return the input rR DBFunctorList of source column names DBFunctorList of target column names DBFunctorColumn Transformation function DBFunctorRemove source column option DBFunctorFiltering predicate DBFunctorOutput Column Mapping" DBFunctor input RTable DBFunctor output RTable# DBFunctor input RTable1 DBFunctorinput RTable2  DBFunctor output RTableV DBFunctor!ETL operation of this ETL mapping DBFunctor input RTable DBFunctoroutput ETLMappingW DBFunctor!ETL operation of this ETL mapping DBFunctor input RTable1 DBFunctor input RTable2 DBFunctoroutput ETLMappingX DBFunctor!ETL operation of this ETL Mapping DBFunctor[Right RTable (right branch) (if this is a Unary ETL mapping this should be an emptyRTable)  DBFunctor*Previous ETL mapping (left branch)  DBFunctor8New ETL Mapping, which has added at the end the new node$ DBFunctorinput ETLMapping DBFunctor output RTable empty ETL mappingZ DBFunctorinput ETLMapping DBFunctoroutput RTabResult/-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[/?@ABCDEFGHIJKLMN>R9:;<=-./012345678OPQSTUYZ[VWX<A simple Embedded DSL for ETL/ELT data processing in Haskell(c) Nikos Karagiannidis, 2018 BSD3 nkarag@gmail.com stable POSIX NonezN% DBFunctorInternal type: We use this data type in order to identify unary vs binary operations and if the table is coming from the left or right branch_ DBFunctorThe Set sub-clause of an  rI clause. It specifies each column to be updated along with the new value.a DBFunctor?A grouping predicate clause. It defines an arbitrary function (RGroupPRedicate), which drives when two q"s should belong in the same group.c DBFunctorsDefines the name of the column that will hold the aggregate operation result. It resembles the "AS" clause in SQL.e DBFunctor6Julius Clause to provide a custom aggregation functiong DBFunctor3These are the available aggregate operation clausesi DBFunctorCount aggregation (no distinct)j DBFunctor"Count distinct aggregation (i.e., count(distinct col)@ in SQL). Returns the distinct number of values for this column.k DBFunctorReturns the number of q s in the r (i.e., count(*) in SQL) n DBFunctorAverage aggregationo DBFunctorString aggregationp DBFunctorA custom aggregate operationq DBFunctorAn Aggregate Operation Clauses DBFunctor+Join Predicate Clause. It defines when two qs should be paired.u DBFunctorWThis clause is used for expressions where we do not allow the use of the Previous valuew DBFunctorA Table Expression defines the r< on which the current ETL Operation will be applied. If the y! constructor is used, then this rP is the result of the previous ETL Operations in the current Julius Expression ()z DBFunctor3Resembles the "FROM" clause in SQL. It defines the r2 on which the Relational Operation will be applied| DBFunctor9It is used to define an arbitrary binary operation on an r~ DBFunctor8It is used to define an arbitrary unary operation on an r DBFunctor Subclause on  clause. Defines the source q of the insert operation. DBFunctor$This subclause refers to the source r that will feed an  operation DBFunctor"Update columns subclause of Upsert DBFunctor When Matched subclause of Upsert DBFunctor#Upsert matching condition subclause DBFunctor)Upsert source subclause (Using clause in SQL) DBFunctorMerge Into subclause DBFunctor Subclause on : clause. Defines the source of the insert operation. The Values& branch is used for inserting a singl q , while the RTuples' branch is used for inserting a whole rd, typically derived as the result of a Julius expression. The former is similar in concept with an INSERT INTO VALUES< SQL clause, and the latter is similar in concept with an INSERT INTO SELECT SQL clause. DBFunctorInsert Into subclause DBFunctorThe Relational Operation (') is a Julius clause that represents a Relational Algebra Operation.  DBFunctorqS filtering clause (selection operation), based on an arbitrary predicate function (B) DBFunctorColumn projection clause DBFunctorAggregate Operation clause DBFunctorDGroup By clause, based on an arbitrary Grouping predicate function () DBFunctorYInner Join clause, based on an arbitrary join predicate function - not just equi-join - (") DBFunctorXLeft Join clause, based on an arbitrary join predicate function - not just equi-join - (") DBFunctorYRight Join clause, based on an arbitrary join predicate function - not just equi-join - (") DBFunctor^Full Outer Join clause, based on an arbitrary join predicate function - not just equi-join - (") DBFunctoroImplements the semi-Join operation between two RTables (any type of join predicate is allowed) It returns the qs from the left r that match with the right r. Note that if an q from the left r matches more than one qs from the right r4 the semi join operation will return only a single q.  DBFunctoroImplements the anti-Join operation between two RTables (any type of join predicate is allowed) It returns the qs from the left r that DONT match with the right r. DBFunctorIntersection clause DBFunctor7Union clause. Note this operation eliminates dublicate  DBFunctor<Union All clause. It is a Union operation without dublicate q elimination. DBFunctor'Minus clause (set Difference operation) DBFunctor/This is a generic unary operation on a RTable ($;). It is used to define an arbitrary unary operation on an r DBFunctor0This is a generic binary operation on a RTable (#<). It is used to define an arbitrary binary operation on an r DBFunctorOrder By clause. DBFunctorDelete operation. Deletes the q s from an r based on an B. Please note that this is an  immutable implementation of an rD update. This simply means that the delete operation returns a new r. So, the original rx remains unchanged and no deletion in-place takes place whatsoever. Moreover, if we have multiple threads deleting an r?, due to immutability, each thread "sees" its own copy of the r3 and thus there is no need for locking the deleted q s, as happens in a common RDBMS. DBFunctor Update an r . Please note that this is an  immutable implementation of an rD update. This simply means that the update operation returns a new r that includes all the qs of the original rW, both the ones that have been updated and the others that have not. So, the original rv remains unchanged and no update in-place takes place whatsoever. Moreover, if we have multiple threads updating an r?, due to immutability, each thread "sees" its own copy of the r3 and thus there is no need for locking the updated q s, as happens in a common RDBMS. DBFunctor(Insert Operation. It can insert into an r a single q or a whole r&. The latter is the equivalent of an INSERT INTO SELECT clause in SQL. Since, an r can be the result of a Julius expression (playing the role of a subquery within the Insert clause, in this case). Please note that this is an  immutable implementation of an rD insert. This simply means that the insert operation returns a new r# and does not affect the original r). Also note that the source and target rDs should have the same structure. By "structure", we mean that the oQs and the corresponding data types must match. Essentially what we record in the C must be the same for the two rs. Otherwise a  exception will be thrown. DBFunctorAUpsert (Update+Insert, aka Merge) Operation. We provide a source r and a matching condition ( ) to the qs of the target r. An q from the target r might match to a single only q in the source r;, or not match at all. If it is matched to more than one qs then an exception ()is thrown. When an q from the target r is matched to a source q/, then the corresponding columns of the target q9 are updated with the new values provided in the source q". This takes place for the target q/s that match but also that satisfy the input B1. Thus we can restrict further with a filter the qs of the target r8 where the update will take place. Finally, the source q#s that did not match to the target r(, are inserted (appended) to the target rPlease note that this is an  immutable implementation of an rD upsert. This simply means that the upsert operation returns a new r# and does not affect the original r(. Also note that the source and target rDs should have the same structure. By "structure", we mean that the oQs and the corresponding data types must match. Essentially what we record in the C must be the same for the two rs. Otherwise a  exception will be thrown.  An Example: Source RTable: srcTab = Id | Msg | Other ----|---------------|------- 1 | "updated" |"a" 2 | "world2" |"a" 3 | "inserted" |"a" Target RTable: trgTab = Id | Msg | Other ----|---------------|------- 1 | "hello1" |"b" 2 | "world1" |"b" 4 | "old" |"b" 5 | "hello" |"b" juliusToRTable $ EtlMapStart :-> (EtlR $ ROpStart :.(Upsert $ MergeInto (Tab trgTab) $ Using (TabSrc srcTab) $ MergeOn (RUpsertPredicate ["Id"] (\t1 t2 -> t1 <!> "Id" == t2 <!> "Id")) $ -- merge condition: srcTab.Id == trgTab.Id WhenMatchedThen $ UpdateCols ["Msg"] $ FilterBy (\t -> let msg = case toText (t <!> "Msg") of Just t -> t Nothing -> pack "" in (take 5 msg) == (pack "hello") ) -- Msg like "hello%" ) ) Result RTable: Id | Msg | Other ----|---------------|------- 1 | "updated" |"b" -- Updated RTuple. Note that only column "Msg" has been overwritten, as per the UpdateCols subclause 2 | "world1" |"b" -- Not affected due to FilterBy predicate 3 | "inserted" |"a" -- Inserted RTuple 4 | "old" |"b" -- Not affected due to MergeOn condition 5 | "hello" |"b" -- Not affected due to MergeOn condition  DBFunctor#A Relational Operation Expression (P) is a sequence of one or more Relational Algebra Operations applied on a input r6. It is a sub-expression within a Julius Expression () and we use it whenever we want to apply relational algebra operations on an RTable (which might be the result of previous operations in a Julius Expression). A Julius Expression (&) can contain an arbitrary number of 's. The relational operation connector " is left associative because in a @ operations are evaluated from left to right (or top to bottom). DBFunctor Predicate for Deletion Operation DBFunctorAn q predicate clause. DBFunctor^Indicator of whether the source column(s) in a Column Mapping will be removed or not (used in C) If a target-column has the same name with a source-column and a  has been specified, then the (target-column, target-value) key-value pair, overwrites the corresponding (source-column, source-value) key-value pair. DBFunctor Defines the r/ that the current operation will be applied to. DBFunctorKDefines the column transformation function of a Column Mapping Expression ( ), the input r: that this transformation will take place, an indicator (B) of whether the Source Columns will be removed or not in the new rL that will be created after the Column Mapping is executed and finally, an q filter predicate () that defines the subset of qLs that this Column Mapping will be applied to. If it must be applied to all q!s, then for the last parameter (%), we can just provide the following B: FilterBy (\_ -> True)  DBFunctor;Defines the Target Columns of a Column Mapping Expression (*) and the column transformation function (>). DBFunctorA Column Mapping (?) is the main ETL/ELT construct for defining a column-level transformation. Essentially with a Column Mapping we can create one or more new (derived) column(s) (Target Columns2), based on an arbitrary transformation function (>6) with input parameters any of the existing columns (Source Columns ). So a  is either empty, or it defines the source columns, the target columns and the transformation from source to target. Notes: * If a target-column has the same name with a source-column and a , or a  has been specified, then the (target-column, target-value) key-value pair, overwrites the corresponding (source-column, source-value) key-value pair * The returned r will include only the q5s that satisfy the filter predicate specified in the  clause. DBFunctor4A named intermediate result in a Julius expression (), which we can access via the  function. DBFunctorTThe name of an intermediate result, used as a key for accessing this result via the  function. DBFunctorCAn ETL Operation Expression is either a Column Mapping Expression ()), or a Relational Operation Expression () DBFunctorAn ETL Mapping Expression is a "Julius Expression"j. It is a sequence of individual ETL Operation Expressions. Each such ETL Operation "acts" on some input r) and produces a new "transformed" output r. The ETL Mapping connector  (as well as the - connector) is left associative because in a c operations are evaluated from left to right (or top to bottom) A Named ETL Operation Expression () is just an ETL Operation with a name, so as to be able to reference this specific step in the chain of ETL Operations. It is actually a named intermediate resultH, which we can reference and use in other parts of our Julius expression DBFunctor/Returns a prefix of an ETLMappingExpr that matches a named intermediate result. For example, below we show a Julius expression where we define an intermediate named result called "myResult". This result, is used at a later stage in this Julius expression, with the use of the function takeNamedResult. g etlXpression = EtlMapStart :-> (EtlC $ ...) :=> NamedResult "myResult" (EtlR $ ...) :-> (EtlR $ ... ) :-> (EtlR $ ROpStart :. (Minus (TabL $ juliusToRTable $ takeNamedResult "myResult" etlXpression -- THIS IS THE POINT WHERE WE USE THE NAMED RESULT! ) (Previous)) ) In the above Julius expression (etlXpresion) the "myResult" named result equals to the prefix of the etlXpresion, up to the operation (included) with the named result "myResult".  takeNamedResult "myResult" etlXpression == EtlMapStart :-> (EtlC $ ...) :=> NamedResult "myResult" (EtlR $ ...) Note that the julius expression is scanned from right to left and thus it will return the longest prefix expression that matches the input name DBFunctor9Evaluates (parses) the Julius exrpession and produces an -. The -^ is an internal representation of the Julius expression and one needs to combine it with the Y> function, in order to evaluate the Julius expression into an r.. This can be achieved directly with function  DBFunctorjPure code to evaluate the "ETL-logic" of a Julius expression and generate the corresponding target RTable./The evaluation of a Julius expression (i.e., a ?) to an RTable is strict. It evaluates fully to Normal Form (NF) as opposed to a lazy evaluation (i.e., only during IO), or evaluation to a WHNF. This is for efficiency reasons (e.g., avoid space leaks and excessive memory usage). It also has the impact that exceptions will be thrown at the same line of code that 2 is called. Thus one should wrap this call with a & handler, or use , or 2, if one wants to handle the exception gracefully.Example: do catch (printRTable $ juliusToRTable $ <a Julius expression> ) (\e -> putStrLn $ "There was an error in the Julius evaluation: " ++ (show (e::SomeException)) )  Or, similarly Bdo p <- (eitherPrintRTable printRTable $ juliusToRTable $ <a Julius expression> ) :: IO (Either SomeException ()) case p of Left exc -> putStrLn $ "There was an error in the Julius evaluation: " ++ (show exc) Right _ -> return ()  DBFunctorEvaluate a Julius expression within the IO Monad. I.e., Effectful code to evaluate the "ETL-logic" of a Julius expression and generate the corresponding target RTable./The evaluation of a Julius expression (i.e., a ?) to an RTable is strict. It evaluates fully to Normal Form (NF) as opposed to a lazy evaluation (i.e., only during IO), or evaluation to a WHNF. This is for efficiency reasons (e.g., avoid space leaks and excessive memory usage). It also has the impact that exceptions will be thrown at the same line of code that 2 is called. Thus one should wrap this call with a & handler, or use 2, if he wants to handle the exception gracefully.Example: do result <- catch (runJulius $ <a Julius expression>) (e -> do putStrLn $ "there was an error in Julius evaluation: " ++ (show (e::SomeException)) return emptyRTable )  DBFunctorAEvaluate a Julius expression and return the corresponding target r or an exception. One can define custom exceptions to be thrown within a Julius expression. This function will catch any exceptions that are instances of the ' type class./The evaluation of a Julius expression (i.e., a ) to an r is strict. It evaluates fully to Normal Form (NF) as opposed to a lazy evaluation (i.e., only during IO), or evaluation to a WHNF. This is for efficiency reasons (e.g., avoid space leaks and excessive memory usage). Example: ldo res <- (eitherRunJulius $ <a Julius expression>) :: IO (Either SomeException RTable) resultRTab <- case res of Right t -> return t Left exc -> do putStrLn $ "there was an error in Julius evaluation: " ++ (show exc) return emptyRTable  DBFunctorEReceives an input Julius expression, evaluates it to an ETL Mapping (-+) and executes it, in order to return an  containing an rA storing the result of the ETL Mapping, as well as the number of q s returned  DBFunctor?Evaluate a Julius expression within the IO Monad and return an . DBFunctorFEvaluate a Julius expression within the IO Monad and return either an 9, or an exception, in case of an error during evaluation. DBFunctorKGeneric ETL execution function. It receives a list of input (aka "source") rEs and an ETL function that produces a list of output (aka "target") rQs. The ETL function should embed all the "transformation-logic" from the source rs to the target rs. DBFunctorFGeneric ETL execution function that returns either the target list of rus, or an exception in case of a problem during the ETL code execution. It receives a list of input (aka "source") rEs and an ETL function that produces a list of output (aka "target") rQs. The ETL function should embed all the "transformation-logic" from the source rs to the target rs.( DBFunctorAEvaluates (parses) a Relational Operation Expression of the form   ROpStart :. ROp ... :. ROp 9and produces the corresponding ROperation data type. This will be an RCombinedOp relational operation that will be the composition of all relational operators in the ROpExpr. In the returned result the TabExpr corresponding to the left and right RTable inputs to the ROperation respectively, are also returned.) DBFunctorQturns the list of agg operation expressions to a list of RAggOperation data type DBFunctor Returns an 9- that adds a surrogate key (SK) column to an r and fills each row with a SK value. This function is only exposed for backward compatibility reasons. The recommended function to use instead is ?, which can be embedded directly into a Julius expression as a $. DBFunctor Returns an $ (r -> r.) that adds a surrogate key (SK) column to an rt and fills each row with a SK value. It primarily is intended to be used within a Julius expression. For example: 9 GenUnaryOp (On Tab rtab1) $ ByUnaryOp (addSurrogateKeyJ TxSK 0)  DBFunctor Returns an 9 that Appends an r to a target rq This function is only exposed for backward compatibility reasons. The recommended function to use instead is ?, which can be embedded directly into a Julius expression as a #. DBFunctor Returns a # (r -> r -> r) that Appends an r to a target rO. It is primarily intended to be used within a Julius expression. For example: D GenBinaryOp (TabL rtab1) (Tab $ rtab2) $ ByBinaryOp appendRTableJ  DBFunctor#the name of the intermediate result DBFunctorinput ETLMapping Expression DBFunctoroutput ETLMapping Expression DBFunctorThe name of the surrogate key column -> Integer -- ^ The initial value of the Surrogate Key will be the value of this parameter  DBFunctorNThe initial value of the Surrogate Key will be the value of this parameter  DBFunctorLOutput ETL operation which encapsulates the add surrogate key column mapping DBFunctor$The name of the surrogate key column DBFunctorNThe initial value of the Surrogate Key will be the value of this parameter  DBFunctor Input RTable DBFunctor Output RTable DBFunctorOutput ETL Operation DBFunctor Target RTable DBFunctor Input RTable DBFunctorOutput RTable y   !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ`[\]^_abcdefgmihjklnopqrstuvwxyz{|}~_`abcdefghmlijknopqrstuvwxyz{|}~wwxy_`z{qrghmlijknopcdefabuvst~|}655 Implements r3 over CSV (TSV, or any other delimiter) files logic(c) Nikos Karagiannidis, 2018 BSD3 nkarag@gmail.com stable POSIX None"#=?X& DBFunctor/This exception signifies an error in parsing a   to an Z value DBFunctor<Exception to signify an error in decoding a CSV file into a  data type DBFunctorQOptions for a CSV file (e.g., delimiter specification, header specification etc.) DBFunctorYes or No sum type DBFunctorDefinition of a CSV column. DBFunctorJDefinition of a CSV Row. Essentially a Row is just a Vector of ByteString DBFunctorCDefinition of a CSV file. Treating CSV data as opaque byte strings DBFunctor.reads a CSV file and returns a lazy bytestring DBFunctorreads a CSV file and returns a 5 data type (Treating CSV data as opaque byte strings) DBFunctorTreads a CSV file based on input options (delimiter and header option) and returns a 5 data type (Treating CSV data as opaque byte strings) DBFunctor4write a CSV (bytestring) to a newly created csv file DBFunctorwrite a  to a newly created csv file DBFunctorprint input CSV on screen DBFunctor print input  on screen DBFunctor0copy input csv file to specified output csv file* DBFunctorcsvToRTable: Creates an RTable from a CSV and a set of RTable Metadata. The RTable metadata essentially defines the data type of each column so as to call the appropriate data constructor of RDataType and turn the ByteString values of CSV to RDataTypes values of RTable We assume that the order of the columns in the CSV is identical with the order of the columns in the RTable metadata+ DBFunctor4rtableToCSV : Retunrs a CSV from an RTable The first line of the CSV will be the header line, taken from the RTable metadata. Note that the driver for creating the output CSV file is the input RTableMData descrbing the columns and RDataTypes of each RTuple. This means, that if the RTableMData include a subset of the actual columns of the input RTable, then no eror will occure and the output CSV will include only this subset. In the same token, if in the RTableMData there is a column name that is not present in the input RTable, then an error will occur. DBFunctor creates a    (as defined in Data.Csv ) from an r DBFunctorO(1) First row DBFunctorKO(1) Yield all but the first row without copying. The CSV may not be empty. DBFunctor6selectNrows: Returns the first N rows from a CSV file DBFunctorkColumn projection on an input CSV file where desired columns are defined by position (index) in the CSV. DBFunctor/CSV data are "Tabular" data thus implement the s interface  DBFunctor the CSV file DBFunctorthe output CSV DBFunctor the CSV file DBFunctorthe output CSV type DBFunctor the CSV file DBFunctorthe output CSV type DBFunctorthe csv file to be created DBFunctor input CSV DBFunctorthe csv file to be created DBFunctorinput  DBFunctor!input CSV to be printed on screen DBFunctorinput  to be printed on screen DBFunctorinput csv file DBFunctoroutput csv file+ DBFunctor+input RTable metadata describing the RTable DBFunctor input RTable DBFunctor output CSV DBFunctorNumber of rows to select DBFunctor Input csv  DBFunctor Output csv DBFunctorinput list of column indexes DBFunctor input csv DBFunctor output CSVuttu,   !!"#$%&&'()*+,-./0123456789:;<=>?@ABCDEFGHIJJKLMNOOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~      !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijjklmnopqrstuvwxyz{|}~TUV         !"#$%&'()*+, -. /012345(DBFunctor-0.1.1.0-7Lx0zfYxtD0CRZBriX9SCp RTable.CoreEtl.Internal.Core Etl.JuliusRTable.Data.CSVPaths_DBFunctor Data.HashMapStrictDataListData.CsvHeaderbase Control.Monad<=<GHC.Base.UniquenessViolationInUpsertConflictingRTableStructuresEmptyInputStringsInToRTimestampUnsupportedTimeStampFormatColumnDoesNotExistFormatSpecifier DefaultFormatFormat ColFormatMap RTupleFormat colSelectList colFormatMap RTabResult RTuplesRet Delimiter AggFunction RAggOperation sourceCol targetColaggFuncRGroupPredicateRUpsertPredicate matchCols matchPredRJoinPredicateBinaryRTableOperationUnaryRTableOperation OrderingSpecAscDesc ROperationROperationEmptyRUnionRInterRDiffRPrjRFilterRInJoin RLeftJoin RRightJoin RSemiJoin RAntiJoin RAggregateRGroupBy RCombinedOpRBinOpROrderBy colPrjListfpredjpredaggListgpred colGrByListrcombOprbinOp colOrdList RPredicate ColumnInfonamedtype ColumnOrder RTupleMData RTableMDatartname rtuplemdata pkColumns uniqueKeys RTimestamp RTimestampValyearmonthdayhours24minutesseconds IgnoreDefaultIgnore NotIgnore RDataTypeRIntRTextRDateRTimeRDoubleNullrintrtextrdatedtformatrtimerdouble ColumnDType UknownTypeIntegerVarcharDate TimestampDouble RTableName ColumnNameNameRTupleRTableRTabulartoRTable fromRTable rtableToListrtableFromList rtupleToListrtupleFromListisNull isNotNull stdDateFormatgetColumnNamesFromRTabgetColumnNamesFromRTuplegetColumnInfoFromRTabgetColumnInfoFromRTuple getTheTypeheadRTup rtupLookuprtupLookupDefaultgetRTupColValuenvl nvlColValuedecodeColValue nvlRTuple nvlRTable decodeRTable stripRText rdtappendremoveCharAroundRText toRTimestamp toUTCTime fromUTCTimeinstr instrText instrRTextcreateRTimestamptoTextfromTextisTextstdTimestampFormatrTimestampToRTextcreateRTableMDatatoListColumnNametoListColumnInfotoListRDataTypelistOfColInfoRDataTypecreateRDataTyperaggGenericAgg raggStrAggraggSum raggCount raggCountStar raggCountDistraggAvgraggMaxraggMinropUrunUnaryROperationropUresrunUnaryROperationResropBrunBinaryROperationropBresrunBinaryROperationRes isRTabEmpty isRTupEmpty emptyRTable emptyRTuplecreateSingletonRTable createRTuplecreateNullRTuple isNullRTuple rtabFoldr'rdatatypeFoldr' rtabFoldl'rdatatypeFoldl'rtabMap rtupleMaprtupleMapWithKey rtuplesRet getRTuplesRet rtabResult runRTabResultexecRTabResult removeColumn addColumnf runRfilterp runProjectionrunProjectionMissedHitslimitaJ runAntiJoinsJ runSemiJoiniJ runInnerJoinO joinRTupleslJ runLeftJoingetUniqueColumnNamesAfterJoinrJ runRightJoinfoJrunFullOuterJoinurunUnion runUnionAlli runIntersectdrunDiffrAggrunAggregationrO runOrderByrGgroupNoAggList concatRTab groupNoAgg runGroupByrCombrunCombinedROpinsertAppendRTabinsertPrependRTabinsertRTabToRTab upsertRTabrtabsSameStructurertuplesSameStructure deleteRTab updateRTab updateRTuple upsertRTuplegenDefaultColFormatMapgenColFormatMapgenRTupleFormatgenRTupleFormatDefaulteitherPrintfRTable printfRTableeitherPrintRTable printRTable$fOrdRTimestamp$fEqRTimestamp$fNFDataRTimestamp$fFractionalRDataType$fNumRDataType$fOrdRDataType $fEqRDataType$fNFDataRDataType$fExceptionColumnDoesNotExist%$fExceptionUnsupportedTimeStampFormat*$fExceptionEmptyInputStringsInToRTimestamp&$fExceptionConflictingRTableStructures&$fExceptionUniquenessViolationInUpsert$fShowColumnDType$fEqColumnDType$fEqIgnoreDefault$fShowIgnoreDefault$fShowRTimestamp$fReadRTimestamp$fGenericRTimestamp$fShowRDataType$fReadRDataType$fGenericRDataType$fShowColumnInfo$fEqColumnInfo$fShowRTableMData$fEqRTableMData$fShowOrderingSpec$fEqOrderingSpec$fEqFormatSpecifier$fShowFormatSpecifier$fEqRTupleFormat$fShowRTupleFormat$fEqColumnDoesNotExist$fShowColumnDoesNotExist$fEqUnsupportedTimeStampFormat $fShowUnsupportedTimeStampFormat#$fEqEmptyInputStringsInToRTimestamp%$fShowEmptyInputStringsInToRTimestamp$fEqConflictingRTableStructures!$fShowConflictingRTableStructures$fEqUniquenessViolationInUpsert!$fShowUniquenessViolationInUpsert ETLMapping ETLMapEmptyETLMapLDETLMapRD ETLMapBaletlOptabLtabRtabLrdtabRrdtabLbaltabRbal ETLOperationETLrOpETLcOpropcmapColXForm RColMapping ColMapEmptyRMap1x1RMapNx1RMap1xNRMapNxMsrcCol removeSrcColtrgCol transform1x1srcRTupleFilter srcColGrp transformNx1 trgColGrp transform1xN transformNxMYesNoYesNocreateColMappingrunCMetlOpUetlOpBcreateLeafETLMapLDcreateLeafBinETLMapLDconnectETLMapLDetletlResrtabToETLMapping$fEqETLMapping $fEqYesNo $fShowYesNo SetColumnsSet GroupOnPredGroupOnAsColumnAsAggByAggOpSumCount CountDist CountStarMinMaxAvgStrAggGenAgg AggregateAggOn TabExprJoinJoinOn TabLiteralTabLTabExprTabPrevious FromRTableFromByGenBinaryOperation ByBinaryOpByGenUnaryOperation ByUnaryOp ValuesClause TabSourceTabSrc UpdateColumns UpdateCols WhenMatchedWhenMatchedThenMergeMatchConditionMergeOn MergeSourceUsing MergeInto InsertSourceValuesRTuples IntoClauseInto RelationalOpFilterSelectAggGroupByJoinLJoinRJoinFOJoinSemiJoin SemiJoinPAntiJoin AntiJoinP IntersectUnionUnionAllMinusMinusP GenUnaryOp GenBinaryOpOrderByDeleteUpdateInsertUpsertROpExprROpStart:. ByDelPredWhereByPredFilterBy RemoveSrcCol RemoveSrc DontRemoveSrcOnRTableOn ByFunctionByToColumnTargetColMappingExprSourceColMappingEmptyNamedMap NamedResultNamedResultName ETLOpExprEtlCEtlRETLMappingExpr EtlMapStart:->:=>takeNamedResult evalJuliusjuliusToRTable runJuliuseitherRunJuliusjuliusToResultrunJuliusToResulteitherRunJuliusToResultrunETL eitherRunETLaddSurrogateKeyaddSurrogateKeyJ appendRTable appendRTableJCSVColumnToRDataTypeErrorCsvFileDecodingError CSVOptions delimiter hasHeaderColumnRowCSVcsv readCSVFilereadCSVreadCSVwithOptions writeCSVFilewriteCSV printCSVFileprintCSVcopyCSVcsvHeaderFromRtableheadCSVtailCSV selectNrowsprojectByIndex $fRTabularCSV$fExceptionCsvFileDecodingError$$fExceptionCSVColumnToRDataTypeError$fEqCsvFileDecodingError$fShowCsvFileDecodingError$fEqCSVColumnToRDataTypeError$fShowCSVColumnToRDataTypeErrorversion getBinDir getLibDir getDynLibDir getDataDir getLibexecDir getSysconfDirgetDataFileNameString Text.Printfprintfghc-prim GHC.TypesTrue4unordered-containers-0.2.10.0-LgoTL3wbBEY5bZIDJiyxW4Data.HashMap.BaseHashMap time-1.8.0.2 Data.Time.Clock.Internal.UTCTimeUTCTime GHC.MaybeNothing text-1.2.3.1Data.Text.InternalTextcreateRTupleMdata strAggFoldsumFold countFold countStarFold countDistFoldmaxFoldminFoldFalse runInnerJoincreateOrderingSpec Data.EitherEithergetMaxLengthPerColumnFmtgetMaxLengthPerColumnaddSpace addCharacterprintContLineFmt printContLineprintRTableHeaderFmtprintRTableHeaderprintRTupleFmt printRTuplerdataTypeToStringFmtrdataTypeToString runColMappingrunUnaryETLOperationrunBinaryETLOperation runETLmappingTabExprEnhancedGHC.IOcatchGHC.Exception.Type Exception evalROpExpraggOpExprToAggOp csvToRTable rtableToCSV