NEWS

emuR 2.5.1

new features / performance tweaks / improvements

bug fixes

emuR 2.5.0 (2024-03-15)

new features / performance tweaks / improvements

CRUD operations: create_itemsInLevel() can now be used in more ways than before
CRUD operations: Improved documentation for create/update/delete_itemsInLevel(); this is ongoing work

bug fixes

remove one instance of is.R() because it is no longer allowed on CRAN
CRUD operations: delete_itemsInLevel() was heavily flawed and is now usable

emuR 2.4.2 (2023-11-03)

new features / performance tweaks / improvements

experimental: signal tracks can be stored in Rda format as well as SSFF

bug fixes

passing a data frame or tibble as language option to BAS Web Services calls was broken (closes #272, thanks to @johannacronenberg)
fix error in create_itemsInLevel where end of last segment in a level could be miscalculated

emuR 2.4.1 (2023-09-27)

new features / performance tweaks / improvements

bug fixes

worked around a problem in vroom_write_lines that could materialize on Windows in rare cases
fix loading databases with empty sessions (thanks to @FredrikKarlssonSpeech via #268, #271)

emuR 2.4.0 (2023-05-25)

new features / performance tweaks / improvements

cache for onTheFly computations in get_trackData() (thanks to @samgregory via #254)
improved a small number of error messages

bug fixes

add_files not correctly matching file extension (thanks to @samgregory via #254)
fixed level attribute mixup in generation of seglist (closes #263)
workaround for wrassp::readDataObj() crashing R session. Temporary fix for issue #261.
generic functions Math2/round and Math2/signif did not accept digits parameter for legacy trackdata objects
adaptations to Tidyverse 2.0.0 and other adapations to make unit tests and cran checks smooth again

emuR 2.3.0 (2021-06-11)

new features / performance tweaks / improvements

added explicit checks for all dir.create() (closes #49)
checking dir parameter in import_mediaFiles() (closes #108)
add_files() now works with strings like _annot.json as fileExtension (closes #246)
now prechecking user defined options in .onLoad() (closes #240)
added new sessionPattern and bundlePattern args to list_bundles() and list_sessions() (closes #201)
better output for warning in get_trackdata() regarding heterogeneous sampling rates (closes #189)
using cli package to improve output of summary(emuDBhandle)
slight performance increase in EQL by avoiding full table scans due to REGEXP
checking for repeating sequences in normalize_length() (closes #211)
added explicit error message when bad column name is used in DBconfig (closes #212)
removing linkDefinition on error if it was added be4 (closes #226)

bug fixes

fixed bug in requery workflows like: sl = query(ae, "[Phonetic== V]"); requery_hier(ae, sl[1:2,], level = "Text")
added missing loading of DBconfig to write_bundleList() (fixes #248)
fixed bad handeling of dir paths containgin RegExp chars in convert_TextGridCollection() (fixes #207)

emuR 2.2.0 (2021-03-30)

new features / performance tweaks / improvements

compete rewrite of hierarchical queries + requeries (now using CTEs) + clean-up of the EQL engine
serve() -> save bundle performance bump due to SQLite transaction improvement + using prep. statements
serve() function now downloads current release .zip instead of cloning repo to save time and bandwidth
fixed a few more readr:: file vs path parameter warnings
using readr::write_lines() function in export_TextGridCollection() to better support UTF-8 symbols
added create_spectrogram_image_as_raster function that used the EMU-webApp JavaScript code to draw a spectrogram as a raster

bug fixes

fixed invalid TextGrid export on empty levels

emuR 2.1.1 (2020-09-30)

new features / performance tweaks / improvements

load_emuDB() performance bump due to SQLite transaction improvement + using prep. statements
create_itemsInLevel() now supports items of type SEGMENT
added deprecated/not recommended texts to \code{eplot()}, \code{dplot()} and \code{dur()} help pages (fixes #234)

bug fixes

implemented first version of delete_itemsInLevel()
requery_seq() now returns correct attribute label vs. the level attribute
fixed In rbind(names(params), unlist(params, recursive = F)): number of columns of result is not a multiple of vector length (arg 1) warning in runBASwebservice_maus() with httr::upload_file()
added missing () to return statement
no emuDBcache needed for rename_emuDB()

emuR 2.1.0 (2020-06-15)

new features / performance tweaks / improvements

implemented write_bundleList() and read_bundleList() functions
new bundleListName serve() parameter implemented
added warning to serve() when either bundleComments or bundleFinishedEditing is set to true and no bundleListName was set (closes #268)
implemented onTheFlyFunction parameter for get_trackdata() which allows users to implement their own functions (input: path to wav file; output tibble/data.frame that has a column called "frame_time")
converting factors into characters in get_trackdata() and normalize_length() (closes #224 and #223)
add_perspective() now sets "restrictions" -> "showPerspectivesSidebar" to true to make the side bar visible
normalize_length() now doesn't throw tibble 3.0.0 warnings any more
query() RegEx operators =~ now using SQLites native support (wasn't available in earlier versions)
replaced RCurl with httr dep
moved some dependencies to Suggests field in DESCRIPTION file
removed vignette stubs

bug fixes

try catching rounding errors of sample rate and start time in get_trackdata()

emuR 2.0.4 (2019-11-06)

new features / performance tweaks / improvements

export_seglistToTxtCollection() now zero pads file names (should fix #219)
implemented write_bundleList() and read_bundleList() functions
new bundleListName serve() parameter implemented

bug fixes

using rstudioapi::translateLocalUrl() for ws connection url to permit RStudio to connect
fixed bug in duplicate levels (bad level name insert into items table) that was causing the items array in the JSON file to stay empty
serve() working outside of RStudio again (overlooked that rstudioapi::translateLocalUrl() needs RStudio)
improved error formatting in load_emuDB() and export_TextGridCollection
fixed rename_bundles() issue with sub-string matching of names (fixes #220)
requery_hier() works with non-main attributes again (always returned levels main attribute labels)

emuR 2.0.2 (2019-07-22)

new features / performance tweaks / improvements

changed default resultTypes to "tibble" (query(), requery_hier(), requery_seq(), get_trackdata())
changed url in printed output of serve to https
serve() doesn't block the R console any more
suppressing requery differing length warnings in runBASwebservice_* functions
requery_hier() now persists the input segment list length by inserting NA rows for missing segments
depricated vignettes are now stubs only (== empty containing notice pointing to the EMU-SDMS manual)
implemented rename_bundles() function
better times_norm calculation for normalize_length()
better error message when empty or non existing session is passed into add_files()
serve() useViewer parameter now implemented and the default.
serve() now uses a single server to host the EMU-webApp and provide the websocket server for the emuDB
serve() now works within RStudio when it is run as a web application

bug fixes

emuR 1.1.2 (2019-01-29)

new features / performance tweaks / improvements

rewrite of list_files() that leads to massive speed bump
implemented convert_wideToLong() function to convert trackdata tibble objects to their long form representation (useful for spectral analysis)
query() sub-function now sets perl = TRUE arguments in regexp call when using sessionPattern and bundlePattern (useful for e.g. for negative look-around bundlePattern = "^((?!msajc003).)*$")
implemented export_seglistToTxtCollection()
normalize length now also using list() with do.call(rbind, res_list)
implemted new runMINNI boolean parameter in runBASwebservice_all()

bug fixes

https default on serve()
fixed problems of calcFreqs = T in convert_wideToLong() (possibly caused by newer version of dplyr?)

emuR 1.1.1 (2018-11-12)

new features / performance tweaks / improvements

bug fixes

fixed bad column init. in normalize_length() column
removed detritus LaTex files (e.g. .log, .aux) from vignette/pics directory (as requested by CRAN maintainer)

emuR 1.1.0 (2018-10-26)

new features / performance tweaks / improvements

URL encoding of bundle and session names to allow for URL string reserved characters in bundle and session names
error message of get_trackdata() now contains seglist row index if "Can not extract following"
normalize_length() now allows for additional non-numeric columns
changed stop() to warning() in get_trackdata() when samplerates are inconsistent (closes #190)
better error message when there is a naming mismatch of _emuDB dir _DBconfig.json
using sub() instead of tools::file_path_sans_ext() to handle _ in file extensions
query(), requery_seq() and requery_hier() now supports the resultType "tibble" (and get_trackdata(), requery_seq(), requery_hier() and serve() support them as input)
serve() function now uses seglist$start and seglist$end instead of seglist$sample_start and seglist$sample_end
implemented first version of update_itemsInLevel() (only label updates for now)
now setting the sample_start and sample_end values in query results when EVENT levels are queried (previously only start was set)
implemented first versions of create_links() (currently not checking for anything), create_itemsInLevel() (only EVENTs and ITEMs) and update_itemsInLevel() (only labels) and delete_itemsInLevel()
list_bundles() outside of loop for performance bump in get_trackdata()
better error message in add_files() if no files are found
consistentOutputType of get_trackdata() is not set to TRUE and is reset to T if resultType is "emuRtrackdata" or "tibble" (fixes #203)
avoiding negative times_rel and times_norm values in create_emuRtrackdata() by setting them to 0 (caused by string to numeric conversion precision errors)
requery_seq() now inserts NA values for the out of bounds rows instead of dropping them.
implemented list_sampleRates() function
get_trackdata() is now iteratively appending to a list() instead of into a SQLite temp table. This is a fairly large performance boost and also fixes #206.
added deprecation warnings to vignettes (added links to manual chapters)

bug fixes

propper fix for "now ordering by items_idx not by start_start_seq_idx which led to bad label sequences (fixes #140)"
fixed bad indexing in normalize_length() when sl_rowIdx values are not a c(1, 2, 3, 4, ...) sequence
fixed staticContours SSFF tracks not being sent to EMU-webApp (fixes #195)
fixed bug with completely empty levels that caused a bad resort of levels in _annot.jsons in rewrite_allAnnots()
fixed bug in add_files() that was using the wrong variable (fixes #196)
added error message when querying levels without time-bearing sub-levels (closes #150)
fixed bug in create_emuRtrackdata() with handling trackdata object of class spectral
correct recalculation of ITEM IDs of missing levels in .hlb files (== only present in ESPS files) in convert_legacyEmuDB()
fixed handling of completely empty levels in .hlb files
normalize_length() now handles various additional column types (not just "numeric")

emuR 1.0.0 (2018-04-11)

new features / performance tweaks / improvements

implemented new consistentOutputType parameter for get_trackdata() to always return a trackdata or emuRtrackdata object independent of what the cut and npoints arguments are set to
now removing levelCanvasOrder entry in remove_levelDefinition() (fixes #156)
serve() method now uses GET to deliver media files to the EMU-webApp. This avoids the base64 conversion overhead and is a quite significant load time improvement
explicit error message in convert_legacyEmuDB() when invalid redundant links are found
better error message in BPF parser
convert_legacyEmuDB() automatically converts .ssd media files to .wav and normalizes the annotations to start at 0 (only if attr(ssd,'startTime') is not 0).
added sort() S3 method for emuRsegs objects
checking for badly sorted emuRsegs in requery_hier() and requery_seq() functions
create_emuRtrackdata() returns a simple data.frame object not a data.table object
emuRtrackdata object now contains a times_norm (normalized time values between 0 and 1 for each segment) column by default
added note to print.emuRsegs() to give the user a hint about missing columns
implemented print.emuRtrackdata() to avoid overly verbose output
implemented normalize_length() function as S3 function to normalize the length of each segment in an emuRtrackdata object
added absolute_file_path column to output of list_files()
query engine does not rely on label index in label array any more (updated convert_queryResultToEmuRsegs() to use resultAttrDef instead of labelIdx). Closes #164.
added browser argument to serve() function which is passed on to utils::browseURL() function
requery_seq() now uses start_item_seq_idx and end_item_seq_idx of seglist instead of start_item_id and start_item_id to simplify function
implemented check_emuDBhandle() function that is used on every exported function that takes a emuDBhandle as an argument to check if the handle is still valid (closes #176)
implemented "tibble" as resultType option in get_trackdata(). This will probably replace the "emuRtrackdata" option in future (it contains exactly the same data/columns).
prechecking if attribute definition is already defined (closes #182)
get_trackdata() now uses temporary SQL tables to store the intermediate results (massive performance gains!). Removed nrOfAllocationRows parameter as this is no longer needed as no matrix is used to store the intermediate results. (also closes #125)
convert_TextGridCollectio() using dir.exists() instead of file.exists() to check dirs
all read operations now use the readr package (avoids encoding problems like #187)
list_attributeDefinitions() now allows for a name vector to be passed in
rewrite of rewrite_allAnnots() functions for faster rewrites of _annot.json files to disk
improved cleanup in testing DBconfig functions
now ordering by items_idx not by start_start_seq_idx which led to bad label sequences (fixes #140)

bug fixes

fixed problem of updating cache. Didn't handle data.frame object that was thought to be a vector correctly!
fixed a bug in the BPF export function, which meant that WAVE files were only copied into one session
added missing $ in pattern arguments in list.files call in list_files (fixes #170)
not adding ssffTrackDefinition to DBconfig if user input is no (closes #171)
fixed bad seq_start_seq_idx and seq_seq_idx returned by requery_seq() (fixes #183)
fixed bad seq_start_seq_idx returned by internal query_databaseHier() function
fixed bad sorting of requery_seq()/requery_hier() when calcTimes = F (still sorted by start_sample instead of the correct seq_idx)
added readr::parse_character() to data received in serve() as this is recoded in windows (fixes #188).

emuR 0.2.3 (2017-07-03)

new features / performance tweaks / improvements

tweaked runBASwebservice_maus(); improved performance for presegmented bundles
performance bump for fapply() by preallocating result matrix
performance bump for trapply() by preallocating result matrix
performance bump for mel.spectral() by preallocating result matrix
performance bump for bark.spectral() by preallocating result matrix
updated DBI calls to comply with the latest best practices (using DBI::dbExecute() instead of DBI::dbGetQuery() for non-SELECT queries)
BPF collection exporter documented and now public

bug fixes

export_TextGridCollection() now handles partial includes of bundle and session names correctly (issue #147)
added missing check if anagestConfig is defined to rename_attributeDefinition()
setting useBytes to T to avoid reencoding under windows
fixed bug in add_ssffTrackDefinition() that was trying to access fp which was renamed in a refactor to filesDf
fixed export to autodetect S3 methods (cbind & rbind for trackdata)

emuR 0.2.2 (2017-03-10)

new features / performance tweaks / improvements

some changes to the parameter names in the BAS webservice functions
convert_txtCollection and convert_BPFCollection now name topmost item "bundle"
added functions to set and get level descriptions in DBconfig
BAS webservice functions now perform a cache update prior to departure
added multiple perspectives to ae demo database
choosing explicit paths with intersecting hierarchies now possible
remove levelDef & linkDef now implement force parameters
new function convert_txtCollection converts plain text collections into single-node emuDB
new functions runBASwebservice_* that call various BAS webservices from inside emuR
NULLing out empty DFs on list_level/linkDefs for more consistent API
newLinkDefType argument implemented in autobuild_linkFromTimes() to generate linkDefinition if so desired
automatically removing superlevel from levelCanvasOrder if convertSuperlevel is set to TRUE in autobuild_linkFromTimes()

bug fixes

wrapped readChars in enc2utf8 to avoid encodings issues on windows
updating label table correctly on add_attributeDefinition() (#138)
runBASwebservice_maus / minni / all now no longer ignore unlinked items (idx -1) but treat them as linkless segments
commented out cat() in train() function be be less verbose
BAS webservice calls now get their own temp directories (UUID based). This avoids race conditions when several scripts are running in parallel.
convert_txtCollection now treats perspectives as array (as it should)

emuR 0.2.1 (2016-12-20)

new features / performance tweaks / improvements

added new EMUwebAppConfig -> perspectives -> signalCanvases -> minMaxValLims config option to emuDB vignette
requery_hier + requery_seq now implement the same timeRefSegmentLevel parameter as query (#135)

bug fixes

fixed requery_hier() bug of requery on same attribute definition
fixed requery_hier() bug of requery on same level but different attribute definition

emuR 0.2.0 (2016-11-11)

new features / performance tweaks / improvements

rewrite of query engine to not require links_ext table any more (== redundant links)
calcTimes parameter added to query() / requery_seq() / requery_hier() to make calculating times optional (extreme performance boost if no times have to be calculated)
rewrite of annotJSONcharToBundleAnnotDFs() for faster loads emuDBs containing large annotJSONs
replaced tidyjson as annot.json parser with own solution at tidyjson didn't scale well on larger annotation files
added verbose parameter to export_TextGridCollection()
improved pre-check of dir exists in export_TextGridCollection()
added new replace_itemLabels function
improved export_TextGridCollection() doc
improved replace_itemLables() speed
implemented rename_emuDB() (#116)
implemented duplicate_level() (#113)
implemented linkDuplicates parameter in duplicate_level()
autobuild_linkFromTimes() speed improvements
FUNCQ queries (start(),end(), medial()) now additionally support TRUE & FALSE and T & F values (vs. 0 & 1)
added attrDefNames column to list_levelDefinitions() output
can now deal with read only emuDBs by copying the cache to tempdir() and making it writable for the user
added start_item_seq_idx and end_item_seq_idx to emuRsegs object
added start_item_seq_idx and end_item_seq_idx type values to all intermediate result tables
added optional function to reduce hierarchical query results to left and right most children only (large performance gain on calcTimes = T)
rewriting annot.json files now updates MD5 sums as well (avoids unnecessary reload on next load_emuDB)
rewriting annot.json files now writes all (including empty / missing) attributeDef. labels

bug fixes

fixed bad DBconfig gen. on add_perspective
fixed list_linkDefinitions() returning strings as factors
fixed bad error message when passing in ITEM levels to autobuild_linkFromTimes()
fixed incorrect handling of DBconfig when writeToFS was set to FALSE (writeToFS is now called rewriteAllAnnots)

emuR 0.1.9 (2016-08-04)

new features / performance tweaks / improvements

also allowing "time = " in TextTiers
"levels of type 'EVENT' are not allowed to be super levels (== parents) in a domination relationship" constraint enforced in add_linkDefinition
added "MEDIAFILE_SAMPLES" as constant name to access audio samples to get_trackdata() function
improved error message to include tgPath in create_DBconfigFromTextGrid function
no integer return value returned by create_emuRdemoData() any more! It was implicitly returned from wrassp function call...
improved the slow overlap checking function in the BPF parser (is now O(n) instead of O(n^2))
fixed col naming problems for new (unreleased) RSQLite version
added export_TextGridCollection() function
improved doc for get_trackdata
constant naming of EMU-SDMS vs EMU_SDMS in various files
rewriting all annotation file on add_levelDefinition, remove_levelDefinition

bug fixes

fixed problem in conversion to JSON with empty items array (object '{}' vs array '[]')
fixed problem of keywords "number" | "time" | "xmin" | ... in labels causing TextGrid parser to fail
fixed problem with to lax RegEx in TextGrid parser
fixed validation problem with missing levels regarding types

emuR 0.1.8 (2016-05-03)

new features / performance tweaks / improvements

get_trackdata with onTheFly calculation now reuses AsspDataObj if the current utterance is the same as the previous (large performance gain especially on long audio files)
checking if DBconfig exists for better error message if 'name' field is not set correctly in DBconfig
setting PRAGMA temp_store = 2; for SQLite connections
not extracting tables to R if no RegEx needed to create filtered_tmp tables (performance gain when querying large emuDBs)
convert_BPFCollection can now assigns the same label to more than one item when unifying tiers
newline at the end of load_emuDB if no redundant links are built
queries using dominates operator '^' don't use linksExt table anymore -> large performances benefits
only using _filtered_tmp tables if RegEx patterns are used
changed primary key on items table which leads to massive performance gains (deleting _emuDBcache.sqlite required)

bug fixes

fixed error handling of create_emuRtrackdata + added @export to roxygen doc
invalid annotJSONs generated by import_mediaFiles fixed
convert_TextGridCollection can now handle nested folders again
invalid UUIDs in DBConfig produced by convert_BPFCollection. Also added additional unit test to detect this.
list_bundles uses session argument again
fixed "Expression tree is too large (maximum depth 1000)" error in get_trackdata with long emuRsegs lists

emuR 0.1.7 (2016-03-31)

R depends version bump to 3.2.0 (as requested by CRAN maintainer)
updated testthat::expect_less_than to expect_lt calls (due to deprecated warnings)
Using new .keep_all = T parameter of dplyr
removed legacy version of EQL vignette (overlooked as inst/doc was in .gitignore)

emuR 0.1.6 (2016-03-25)

skipping in-depth thorough tests on CRAN for query and autobuild SQL functions

emuR 0.1.5

fixed problem of interm_res_tables already being present with queries that have multiple recursion depth on both sides of either -> or ^ operand (e.g. query (ae , "[[[Phonetic = n -> Phonetic =z] -> Phonetic = S ] ^ [Text = friends -> Text = she]]"))
fixed bad URL in README.md
added CITATION file

emuR 0.1.3.9000

renamed SQL tables & columns from camel case to underscore notation
variable SQL backend implementation

emuR 0.1.2.9000

multiple check fixes on various platforms

emuR 0.1.1.9000

serve problem with internalVars bug fixed
file locking problem that caused vignettes to fail under windows problem fixed

emuR 0.1.0.9000

massive refactor of all functions that used to refer to an emuDB by name and optionally by its UUID. They now use the new emuDBhandle object that is now returned by the load_emuDB() function.
convert_XXX_to_emuDB() functions renamed to convert_XXX()