Commit Graph

524 Commits (jsx-conditional-rendering)

Author SHA1 Message Date
Uku Taht 5c290b1982 Use stats_start_date field on sites 5 years ago
Uku Taht 0f43f4a9df Remove unused function 5 years ago
Uku Taht 333de87ceb Add stats_start_date field 5 years ago
Uku Taht d40faf6ec7 More datepicker improvements 5 years ago
Uku Taht 7f58e6be4e Add Year to Date option in datepicker 5 years ago
Uku Taht 8467979051 Add All Time option to datepicker 5 years ago
Uku Taht 18fd0dbe04 Always parse GA numbers as floats 5 years ago
Uku Taht ed871e7a0b Parse scientific notation in GA import 5 years ago
Uku Taht 0ab8d653be Add retry mechanism to HTTPoison 5 years ago
Uku Taht ce649966ed Fix display 5 years ago
Uku Taht f00457c5d1 Change enterprise plans 5 years ago
Andrea Mazzarella 3c93a2d91b
1421 contains filter (#1799)
* small refactors and adds 'contains' to modal

* supports contains filter in the backend

* moves entry and exit page under the page filter

* prettier

* updates the CHANGELOG

* undo package-lock changes

* fixes formatting for elixir

* renames unused parameter to _

* Update changelog

* Use uppercase for constants and update type/prefix lookup
5 years ago
Uku Taht c83e038af0 Add fallback if websiteUrl is not defined 5 years ago
RobertJoonas e5cf800d4a
Improve stats api (#1759)
* can now query events metric when breaking down on a visit prop

* updated changelog

* fixed formatting

* fixed a bug - querying for bounce rate does not return visits anymore

* removed the changelog update
5 years ago
RobertJoonas 492f47ba1e
Crm transfer data (#1749)
* pull from master

* added query generation by struct fields

* ready, improved tests

* fixed a naming mistake
5 years ago
Uku Taht 5dc024a5cf Debug API call 5 years ago
Uku Taht a375789f52 Deal with nil when importing 5 years ago
Uku Taht 3a0ec5481b Log import issue 5 years ago
Uku Taht cd7c4bd724 Convert end_date timezone correctly 5 years ago
Uku Taht 51cd24bcaf Allow user to cancel ongoing import 5 years ago
Uku Taht 0f58d56c11 Get real start date for date range 5 years ago
Uku Taht 203f87520b
Ga import improvements (#1784)
* Do not link Google account for import

* Record start date

* Fix tests
5 years ago
Uku Taht ea947a1401 Increase HTTP timeout 5 years ago
Uku Taht 1fb39957a6 Import Oban error handling 5 years ago
Uku Taht 83df555f55 Remove OTEL 5 years ago
Uku Taht e1d6d119b0 Add end date to GA import 5 years ago
Uku Taht c4f7c9a7a0 Add debug loggin to GA import 5 years ago
Uku Taht e026148fb0 Show better error message on duplicate domain
Co-authored-by: Marko Saric <me@markosaric.com>
5 years ago
Uku Taht e27734ed79
[Continued] Google Analytics import (#1753)
* Add has_imported_stats boolean to Site

* Add Google Analytics import panel to general settings

* Get GA profiles to display in import settings panel

* Add import_from_google method as entrypoint to import data

* Add imported_visitors table

* Remove conflicting code from migration

* Import visitors data into clickhouse database

* Pass another dataset to main graph for rendering in red

This adds another entry to the JSON data returned via the main graph API
called `imported_plot`, which is similar to `plot` in form but will be
completed with previously imported data.  Currently it simply returns
the values from `plot` / 2. The data is rendered in the main graph in
red without fill, and without an indicator for the present. Rationale:
imported data will not continue to grow so there is no projection
forward, only backwards.

* Hook imported GA data to dashboard timeseries plot

* Add settings option to forget imported data

* Import sources from google analytics

* Merge imported sources when queried

* Merge imported source data native data when querying sources

* Start converting metrics to atoms so they can be subqueried

This changes "visitors" and in some places "sources" to atoms. This does
not change the behaviour of the functions - the tests all pass unchanged
following this commit. This is necessary as joining subqueries requires
that the keys in `select` statements be atoms and not strings.

* Convery GA (direct) source to empty string

* Import utm campaign and utm medium from GA

* format

* Import all data types from GA into new tables

* Handle large amounts of more data more safely

* Fix some mistakes in tables

* Make GA requests in chunks of 5 queries

* Only display imported timeseries when there is no filter

* Correctly show last 30 minutes timeseries when 'realtime'

* Add with_imported key to Query struct

* Account for injected :is_not filter on sources from dashboard

* Also add tentative imported_utm_sources table

This needs a bit more work on the google import side, as GA do not
report sources and utm sources as distinct things.

* Return imported data to dashboard for rest of Sources panel

This extends the merge_imported function definition for sources to
utm_sources, utm_mediums and utm_campaigns too. This appears to be
working on the DB side but something is incomplete on the client side.

* Clear imported stats from all tables when requested

* Merge entry pages and exit pages from imported data into unfiltered dashboard view

This requires converting the `"visits"` and `"visit_duration"` metrics
to atoms so that they can be used in ecto subqueries.

* Display imported devices, browsers and OSs on dashboard

* Display imported country data on dashboard

* Add more metrics to entries/exits for modals

* make sure data is returned via API with correct keys

* Import regions and cities from GA

* Capitalize device upon import to match native data

* Leave query limits/offsets until after possibly joining with imported data

* Also import timeOnPage and pageviews for pages from GA

* imported_countries -> imported_locations

* Get timeOnPage and pageviews for pages from GA

These are needed for the pages modal, and for calculating exit rates for
exit pages.

* Add indicator to dashboard when imported data is being used

* Don't show imported data as separately line on main graph

* "bounce_rate" -> :bounce_rate, so it works in subqueries

* Drop imported browser and OS versions

These are not needed.

* Toggle displaying imported data by clicking indicator

* Parse referrers with RefInspector

- Use 'ga:fullReferrer' instead of 'ga:source'. This provides the actual
  referrer host + path, whereas 'ga:source' includes utm_mediums and
  other values when relevant.
- 'ga:fullReferror' does however include search engine names directly,
  so they are manually checked for as RefInspector won't pick up on
  these.

* Keep imported data indicator on dashboard and strikethrough when hidden

* Add unlink google button to import panel

* Rename some GA browsers and OSes to plausible versions

* Get main top pages and exit pages panels working correctly with imported data

* mix format

* Fetch time_on_pages for imported data when needed

* entry pages need to fetch bounces from GA

* "sample_percent" -> :sample_percent as only atoms can be used in subqueries

* Calculate bounce_rate for joined native and imported data for top pages modal

* Flip some query bindings around to be less misleading

* Fixup entry page modal visit durations

* mix format

* Fetch bounces and visit_duration for sources from GA

* add more source metrics used for data in modals

* Make sources modals display correct values

* imported_visitors: bounce_rate -> bounces, avg_visit_duration -> visit_duration

* Merge imported data into aggregate stats

* Reformat top graph side icons

* Ensure sample_percent is yielded from aggregate data

* filter event_props should be strings

* Hide imported data from frontend when using filter

* Fix existing tests

* fix tests

* Fix imported indicator appearing when filtering

* comma needed, lost when rebasing

* Import utm_terms and utm_content from GA

* Merge imported utm_term and utm_content

* Rename imported Countries data as Locations

* Set imported city schema field to int

* Remove utm_terms and utm_content when clearing imported

* Clean locations import from Google Analytics

- Country and region should be set to "" when GA provides "(not set)"
- City should be set to 0 for "unknown", as we cannot reliably import
  city data from GA.

* Display imported region and city in dashboard

* os -> operating_system in some parts of code

The inconsistency of using os in some places and operating_system in
others causes trouble with subqueries and joins for the native and
imported data, which would require additional logic to account for. The
simplest solution is the just use a consistent word for all uses. This
doesn't make any user-facing or database changes.

* to_atom -> to_existing_atom

* format

* "events" metric -> :events

* ignore imported data when "events" in metrics

* update "bounce_rate"

* atomise some more metrics from new city and region api

* atomise some more metrics for email handlers

* "conversion_rate" -> :conversion_rate during csv export

* Move imported data stats code to own module

* Move imported timeseries function to Stats.Imported

* Use Timex.parse to import dates from GA

* has_imported_stats -> imported_source

* "time_on_page" -> :time_on_page

* Convert imported GA data to UTC

* Clean up GA request code a bit

There was some weird logic here with two separate lists that really
ought to be together, so this merges those.

* Fail sooner if GA timezone can't be identified

* Link imported tables to site by id

* imported_utm_content -> imported_utm_contents

* Imported GA from all of time

* Reorganise GA data fetch logic

- Fetch data from the start of time (2005)
- Check whether no data was fetched, and if so, inform user and don't
  consider data to be imported.

* Clarify removal of "visits" data when it isn't in metrics

* Apply location filters from API

This makes it consistent with the sources etc which filter out 'Direct /
None' on the API side. These filters are used by both the native and
imported data handling code, which would otherwise both duplicate the
filters in their `where` clauses.

* Do not use changeset for setting site.imported_source

* Add all metrics to all dimensions

* Run GA import in the background

* Send email when GA import completes

* Add handler to insert imported data into tests and imported_browsers_factory

* Add remaining import data test factories

* Add imported location data to test

* Test main graph with imported data

* Add imported data to operating systems tests

* Add imported data to pages tests

* Add imported data to entry pages tests

* Add imported data to exit pages tests

* Add imported data to devices tests

* Add imported data to sources tests

* Add imported data to UTM tests

* Add new test module for the data import step

* Test import of sources GA data

* Test import of utm_mediums GA data

* Test import of utm_campaigns GA data

* Add tests for UTM terms

* Add tests for UTM contents

* Add test for importing pages and entry pages data from GA

* Add test for importing exit page data

* Fix module file name typo

* Add test for importing location data from GA

* Add test for importing devices data from GA

* Add test for importing browsers data from GA

* Add test for importing OS data from GA

* Paginate GA requests to download all data

* Bump clickhouse_ecto version

* Move RefInspector wrapper function into module

* Drop timezone transform on import

* Order imported by side_id then date

* More strings -> atoms

Also changes a conditional to be a bit nicer

* Remove parallelisation of data import

* Split sources and UTM sources from fetched GA data

GA has only a "source" dimension and no "UTM source" dimension. Instead
it returns these combined. The logic herein to tease these apart is:

1. "(direct)" -> it's a direct source
2. if the source is a domain -> it's a source
3. "google" -> it's from adwords; let's make this a UTM source "adwords"
4. else -> just a UTM source

* Keep prop names in queries as strings

* fix typo

* Fix import

* Insert data to clickhouse in batches

* Fix link when removing imported data

* Merge source tables

* Import hostname as well as pathname

* Record start and end time of imported data

* Track import progress

* Fix month interval with imported data

* Do not JOIN when imported date range has no overlap

* Fix time on page using exits

Co-authored-by: mcol <mcol@posteo.net>
5 years ago
RobertJoonas b4992cedc1
Referrer spam blocklist (#1750)
* integrating blocklist library

* loads blocklist dependency from Github
5 years ago
Uku Taht 6bd4b73a91 Add clause for missing date 5 years ago
Uku Taht 8fa1d60ff0 Send cancellation email safely 5 years ago
RobertJoonas 775ae6228f
Streamlining support (#1723)
* can now remove user grace period from crm

* added a created_at column to users and sites in crm
5 years ago
RobertJoonas 6b22806e1f
Allow admin access to locked dashboards (#1710)
* added super-admin access to locked dashboards

* fixed formatting
5 years ago
bytecode1024 d947d761a5
Fix upper limit of range A-Z (#1668)
Since the A-z range includes [, \, ], ^, _, and `, I assume this is a typo on the upper limit of the A-Z range.
5 years ago
Uku Taht 5b893bde39 Revert "Include 0 length pageviews in time_on_page calculation"
This reverts commit cee79e7875.
5 years ago
Uku Taht cee79e7875 Include 0 length pageviews in time_on_page calculation 5 years ago
Uku Taht 52067803df Update session stats from event 5 years ago
ssendev 8077671f8a
Update session screen_size if unknown (#1610) 5 years ago
Uku Taht 0a5f6c5886 Implement wildcard filters for the API 5 years ago
Uku Taht ebb9476b56 Fix visit duration stat 5 years ago
Uku Taht f049e7f358 Fix plan 5 years ago
Uku Taht 50df61cefa Add v3 plans to all_plans 5 years ago
Uku Taht f1e6baa9df Better sentry error 5 years ago
Uku Taht 9a77644517 Better sentry message when plan is not found 5 years ago
Uku Taht 28ab506bb8 Set start time for v3 pricing 5 years ago
Uku Taht 0bf1da09ce New plans 5 years ago
Uku Taht 711cbdfca2 Remove unused custom domain code 5 years ago
Uku Taht 48ad7485c8
PR 1393 continued (#1542)
* Add `utm_content` and `utm_term`.
Support `utm_content` and `utm_term` as requested in #515.

* Add dropdown for UTM options

* Remove utm_content and term from filter modal for now

Co-authored-by: Blender Defender <defenderblender@gmail.com>
5 years ago
Uku Taht 66575f497b Add goal type to JSON response 5 years ago
Uku Taht 97cdc4814e Refactor goals API 5 years ago
Uku Taht 1dba113e2f
[Draft] Improve location translations (#1526)
* WIP

* Use location library for search suggestions

* Remove unused code

* Remove Countries completely

* Fix tests
5 years ago
Uku Taht b49fd19934
Pull 1506 - formatted (#1527)
* adding api route PUT /api/v1/sites/goals with form fields "goal_type" and "goal_value" with supported types "event" and "page"

* arm64 docker images

* adding api route DELETE /api/v1/sites/goals/:goal_id with form param "site_id"

* revert makfile + package.json

* return statement hotfix in case site could not be found

* adding api route PUT /api/v1/sites/goals/:goal_id with form params "site_id", "goal_type", and "goal_value"

* update the goal api routes to accept event_name or page_path instead of goal_value

* cleaning goals model

* mix format

Co-authored-by: Ahmed Abbas <a.abbas@ixdc.net>
5 years ago
Uku Taht 442e401ede Use new location library 5 years ago
RobertJoonas 9022234aa6
added invoices section (#1511)
* added invoices section

* fixed formatting

* UI and code style improvements, paddle sandbox setup only on dev environment (fixed)
5 years ago
Uku Taht a2d0244315 Fix site locked email 5 years ago
Uku Taht 1bc9f6bfb5 Remove more dead code 5 years ago
Uku Taht 01706b7590 Remove dead code 5 years ago
Uku Taht 4d0bc61ffd Remove Twitter stuff 5 years ago
Uku Taht 07af25ffc3 Fix subscription_interval for enterprise plans 5 years ago
Uku Taht 2463296403 Fetch enterprise plan corretly 5 years ago
Uku Taht 37d3f2e36e Make sure that deleting an account also deletes stats 5 years ago
Uku Taht 6aa0d7405d Sample to 20m instead of 10m 5 years ago
Uku Taht 05bf43c1be
City level location data (#1449)
* Merge branch 'plausible_master'

* Add City level details

* Add City level details

* Use ISO codes instead of geoname_id for subdivisions

* Add easier way to configure geolocation database

* Add workflow for dev branch

* Correct clickhouse migration

* Translate subdivision names

* Translate city names

* WIP

* Region and country filters

* Fix region filter

* Remove region_name when removing region filter

* Add modals for regions and cities

* Remove dead code

* WIP

* Revert "WIP"

This reverts commit 3202bf2fe9.

* Feature flag to hide cities when deployed

* Add changelog entry

* Remove unused code

* Remove unused variables

* Fix test

Co-authored-by: AymanTerra <aymanterra@yahoo.com>
5 years ago
Uku Taht 941a91ec22 Make better use of the index on sessions table 5 years ago
Uku Taht 218f1158d6 Show visitor counts in Search Console report
Fixes #1480
5 years ago
Uku Taht e355828465 Remove pageview filter by default 5 years ago
Uku Taht e9cb8eb4e2 Remove grace period if user upgrades 5 years ago
Uku Taht d547d0f996 Send email notification when sites are locked at grace period end 5 years ago
Uku Taht 29cb7462e6 Add grace period to upgrade 5 years ago
Matt Colligan 29e37c8342 More improvements for CSV export (#1446)
* Round time_on_page metric returned by 'pages' API

This rounds the `time_on_page` metric returned as part of the `pages`
API to the nearest second. While this shows no apparent change in the
web UI, it removes the decimal from the exported data in `pages.csv`.

* Tidy up export tests

* Round time_on_page in db query

* Add goal to CSV export tests

* Display CSV export spinner until download is ready

The mechanism used to make the page aware that the download is ready is:

- Client code sets a cookie and requests download.
- Server code handles download and removes cookie when complete.
- Client code polls every 1s to check the cookies, removing spinner when
  the export cookie is removed.
5 years ago
Uku Taht 5c02811d26 Add opentelemetry oban 5 years ago
Uku Taht 76f6a00bec Add honeycomb 5 years ago
Matt Colligan f576fa2a2c
Improvements to CSV export (#1427)
* Add details=True to export API parameters

This makes the ZIP export add `%{"details" => "True"}` to the query's
`params` when fetching data internally for packaging in the ZIP.

This adds bounce_rate and time_on_page to the data in pages.csv, and
bounce_rate and visit_duration to sources.csv.

* Make API return data with consistent names

Some of the data types returned via the JSON or CSV API use inconsistent
naming, and some have redundant name changes (i.e. count -> visitors ->
count). This makes these all consistent and removes the redundancy.

This addresses #1426, fixes some of the CSV headers, and unifies the
JSON and CSV return data labels.

* Update changelog

* Test should use Timex.shift, not relative time

* Return full country names in CSV export

This also replaces the " character with ' in two country names, as those
are the characters used in the names, yielding a more predictable and
'correct' output.

* Fetch CSV exported data concurrently

* Use spinner to indicate when export has started

* Use 300 as default number of brekadown entries for export

Higher numbers (e.g. 1000) seem to cause clickhouse errors when there
many pages to request. It is unclear what is causing the error, as
clickhouse returns an "unknown" error code and an empty error message.
5 years ago
Uku Taht 7364733faf Ensure that question marks are encoded properly for the database 5 years ago
Uku Taht e4bcdfa474 Revert "Get fresh search console data"
This reverts commit 3ab19afea5.
5 years ago
Uku Taht 3ab19afea5 Get fresh search console data 5 years ago
Uku Taht 9edbbe805d Update dependencies 5 years ago
Uku Taht 83b646eaea Do not send slack message on self hosted 5 years ago
Uku Taht 4ee6257fcd Preload when needed 5 years ago
Uku Taht b9b37cdd34 Show pricing v2 when appropriate 5 years ago
Uku Taht 4eae40cd71 Apply usage optimization in background job 5 years ago
Uku Taht 26e34596c8 Allow enterprise plan users to create sites 5 years ago
Uku Taht e66e9bd1b7 Check site limit for enterprise customers 5 years ago
Uku Taht 792e534edd Optimize usage calculation for large accounts 5 years ago
Uku Taht b1594c631e Adjust API key limits for enterprise plans 5 years ago
Uku Taht cc4966a1c3 Change default API limit to 600 5 years ago
Uku Taht 6a5b383e2b Add enterprise plans 5 years ago
Uku Taht 2ca3baeafd Disable email verification by default 5 years ago
Vignesh Joglekar 7e9d83d62e
Adds pagination for custom props & fixes an issue with errant (none) return values (#1382)
* Adds pagination for custom props, fixes errant (none) being returned in the middle of a dataset

* Formatting

* Fixes errant (none) value correctly

* Changelog

* Adds tests
5 years ago
Uku Taht cd68761928 Fix utm_sources when filtering by event:page 5 years ago
Uku Taht bd7de59a9c Show total visitors when filtered for goal 5 years ago
Uku Taht 5be38a57fa Do not return prop names when they are not used 5 years ago
Uku Taht f42f4da44b Fix how sampling is applied when filtering for a goal 5 years ago
RobertJoonas d9a6ac4fd1
Admin - CRM shows site owner (#1345)
* Allow URLs without hostname setting it to (none)

* shows site-owner in crm

* fixed formatting

* removed extra test
5 years ago
Uku Taht c68b88c24b
Upgrade otp (#1313)
* update tool versions to latest

* Upgrades erlang and elixir, format project

Co-authored-by: happysalada <raphael@megzari.com>
5 years ago
Richard Willis 3f8af9a13b
Fix reading config from file (#1312) 5 years ago
Uku Taht 700a65c98a
Remove trial banner for admins & viewers (#1308)
* Start trial only when the user creates a site

* End trial when ownership is transfered
5 years ago
Uku Taht ebc59313a5 Do not match trailing slash by default in page regex 5 years ago
Uku Taht 08afc2ae71 Timeout db queries in 10 seconds instead of 5 5 years ago
Uku Taht 33ba5d79b6
Memoize has_stats? (#1302)
* Memoize has_stats?

* Remove unused alias
5 years ago
Uku Taht 01412d090f Fix sampling in JOIN queries 5 years ago
Uku Taht 77d3d8695d Remove glob filters from API 5 years ago
Uku Taht 0fe053e106 Fix realtime dashboard 5 years ago
Uku Taht bea26f0707 Fix custom prop breakdown when filtering with visit prop 5 years ago
Uku Taht d3ecbd4e17 Fix sampling 5 years ago
Uku Taht 63b7abad58 Disable sampling in the API 5 years ago
Uku Taht 5df15e9169 Remove IO.inspect 5 years ago
Uku Taht 43df90170b Fix compile warnings 5 years ago
Uku Taht f9ea3cda6b Fix custom props in conversions 5 years ago
Uku Taht a9b16fa6c6 Fix exit rate when filtering for page 5 years ago
Uku Taht 3d3601c332 Support negated glob filters everywhere 5 years ago
Uku Taht 49214eaff8 Fix exit pages when using a goal filter 5 years ago
Uku Taht d9ac8ccdb9 Fix samplign in main graph and top pages 5 years ago
Uku Taht 2f98da8f3a Filter for pageviews when no other filter is specified in base.ex 5 years ago
Uku Taht 669866a16b
Finish stats module (#1248)
* Fix small inconsitencies in stats module

* Format
5 years ago
Uku Taht 4fef567623 Fix goal filter 5 years ago
Uku Taht ecbc68f05b Fix entry pages 5 years ago
Uku Taht 79f6756ba2 Make is not filter work 5 years ago
Uku Taht 39f5bc6afb Make sure Direct / None is not showed in small reports 5 years ago
Uku Taht 7d1b1214e2 Implement dashboard with new stats module 5 years ago
Uku Taht 70c6010538 Check lock status after subscription has been updated 5 years ago
Vignesh Joglekar 30ac901141
Adds manual, editable, auto-suggested filters, and negated&globbed path-based filters (#1121)
* Adds manual-filters + friends

commit 308192044d726e9a6f7406e333048ab3407aa260
Merge: 39287ab a299fab
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Fri Jun 11 03:40:55 2021 -0500

    Merge branch 'manual-filters' into manual-filters-2

commit a299fab1fb
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Fri Jun 11 02:20:26 2021 -0500

    Changes to split and pattern matched function for time_on_page

commit 10f10c933a
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Fri Jun 11 01:53:18 2021 -0500

    Fixes a couple of minor UX issues

commit f2e5ce8eb3
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Fri Jun 11 01:49:10 2021 -0500

    Fixes time on page for globbed and negated page paths

commit bb18af6526
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Thu Jun 10 05:24:05 2021 -0500

    Close to finalized version of updated version

    Just needs some additional testing + potentially code cleanup

commit d0b7bfe3da
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Fri May 28 04:21:21 2021 -0500

    Real Dialyzer Fix

commit 296a76af34
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Fri May 28 03:44:29 2021 -0500

    Dialyzer fix

commit 91f3b44017
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Thu May 27 17:11:24 2021 -0500

    Changelog

commit e041f75745
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Thu May 27 17:09:52 2021 -0500

    Formatting

commit f689642204
Merge: e00929b 4ff25f6
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Thu May 27 17:08:47 2021 -0500

    Merge branch 'master' into manual-filters

commit e00929b1c5
Merge: 83887c4 806975e
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Thu May 27 17:01:48 2021 -0500

    Merge branch 'manual-filters' of github.com:Vigasaurus/plausible-analytics into manual-filters

commit 83887c49cc
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Thu May 27 16:59:32 2021 -0500

    Adds tests for suggestions, formats goals suggestion query

commit 1cb7732a08
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Thu May 27 16:36:31 2021 -0500

    Adds goals as auto-complete capable filter

commit 4ca39cc406
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Thu May 27 16:08:22 2021 -0500

    95% completed auto-complete setup

    Still needs:
    - tests
    - goals added as filter

commit 22d38c4cd8
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Thu May 27 03:47:19 2021 -0500

    80% of auto-complete filters progress

    Still needs
    - countries and screen into new format
    - re-style dropdown and background
    - drop debounce time
    - tests

commit 806975ede9
Merge: 81c5e05 1a93542
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Tue May 25 15:28:21 2021 -0500

    Merge branch 'master' into manual-filters

commit 81c5e05760
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Tue May 25 15:21:03 2021 -0500

    Makes colorings on top bar elements consistent

commit fa7f6c2268
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Tue May 25 14:58:25 2021 -0500

    Makes requested changes, adds different version of filter button

commit 7dc65b9b7a
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Sat May 22 04:29:01 2021 -0500

    Changelog

commit c684f1c76a
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Sat May 22 04:26:14 2021 -0500

    Various UI Improvements

    - Makes edit buttons full-length & properly sized
    - Adds remove filter button in edit menu

commit a632e7a8ee
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Sat May 22 03:11:50 2021 -0500

    Adds tests for exclusions and wildcards

commit eb91a7942b
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Sat May 22 03:02:23 2021 -0500

    Fixes editing UX on list view

commit 6209d72aee
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Fri May 21 04:01:17 2021 -0500

    Bugfix in realtime view, formatting

commit 007d44df38
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Fri May 21 03:23:16 2021 -0500

    Second pass - mostly everything user-facing is done

    Still needs:
    - Tests
    - Potentially negating other filters
    - Potentially some code cleanup

commit cb7b5b9fbd
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Fri May 21 01:49:52 2021 -0500

    First pass on manual filter & path regex/negated filters

    Still needs:
    - Form structure on filter modal
    - Edit filter button
    - Filter dropdown UI improvement
    - Filter modal mount data collection
    - Tests
    - Potentially negating other filters

* Fixes an issue between the updated React version and Flatpickr not showing up

Reverting to v16 was the simplest fix without diving into the issue upstream with Flatpickr

* Resolves comments on UI and code style

Co-authored-by: Uku Taht <Uku.taht@gmail.com>
6 years ago
Uku Taht 883841f789 Make sure the latest subscription is always used 6 years ago
Uku Taht e71de6dc1f
Invitations (#1122)
* Invite existing user to a site

* Add invitation flow for non-existing users

* Accept and reject invitations

* Use invitation flow for existing users

* Locking mechanism for sites

* Authorization for site settings

* Show usage based on site ownership

* Add ability to remove members from a site

* Do not show settings link to viewer roles

* Ability to remove invitations

* Remove `Plausible.Sites.count_for/1`

* Fix tests

* Do not show the trial banner after the trial

* Correct trial emails

* Transfer ownership

* Send invitation email to existing user

* Add invitation email flows

* Add plug for role-based authorization

* Rename AuthorizeStatsPlug -> AuthorizeSiteAccess

* Add email flow for ownership transfer

* Fix URLs in emails

* Fix small copy issues

* Make 'People' its own section in site settings

* Notify user via email if their access has been removed

* Check site lock status when invitation is accepted

* Check lock status when user subscribes

* Make sure only admins and owners can create shared links

* Changelog

* Add LockSites to daily cron

* Clean invitations after 48 hours

* Add notices about expiry

* Add invitation expired page

* Add doc link
6 years ago
Uku Taht 3fe8bd6681
Pr/1117 formatted (#1129)
* runtime.exs: get var from file

* Format pr/1117

Co-authored-by: happysalada <raphael@megzari.com>
6 years ago
Uku Taht 39287ab176 Remove IO.inspect 6 years ago
Uku Taht 0054c9fc96 Add api keys to CRM 6 years ago
Uku Taht d9cc21fc06 Add site limit exempt config 6 years ago
Uku Taht ad6faece87
Stats API: Support IN filter expression (#1090)
* Support IN filter expression

* Update clickhouse_ecto
6 years ago
Uku Taht c37a2de627 Revert "Adds manual filters on FE, capability for exclusion filters (and the globbing we're used to) for path-based filters (#1067)"
This reverts commit b6eeb40472.
6 years ago
Uku Taht 28a931ce76 Fix referrer filter 6 years ago
Vignesh Joglekar b6eeb40472
Adds manual filters on FE, capability for exclusion filters (and the globbing we're used to) for path-based filters (#1067)
* First pass on manual filter & path regex/negated filters

Still needs:
- Form structure on filter modal
- Edit filter button
- Filter dropdown UI improvement
- Filter modal mount data collection
- Tests
- Potentially negating other filters

* Second pass - mostly everything user-facing is done

Still needs:
- Tests
- Potentially negating other filters
- Potentially some code cleanup

* Bugfix in realtime view, formatting

* Fixes editing UX on list view

* Adds tests for exclusions and wildcards

* Various UI Improvements

- Makes edit buttons full-length & properly sized
- Adds remove filter button in edit menu

* Changelog

* Makes requested changes, adds different version of filter button

* Makes colorings on top bar elements consistent
6 years ago
Uku Taht 1a93542cd7 Add rate limit to API requests 6 years ago
Andrey Meshkov 90cfed035f
Added parameters for Clickhouse flush interval and max buffer size (#1073)
Currently, both are hard-coded to relatively small values.
This may cause an unnecessary high load on the Clickhouse instance.
It makes sense to make those configurable.
6 years ago
Uku Taht 5ee3167786 Add special plan 6 years ago
Vignesh Joglekar 5f42141e08
Adds time on page as a top stat when a page filter is active (#1051)
* Adds time on page as a top stat when a page filter is active

* Removes dangling debug statement
6 years ago
Vignesh Joglekar 41e4690116
Adds Time on Page metric to Top Pages report (#1007)
* First pass

Needs more testing & potentially cleanup

* Fixes tests, error handling

* Formatting

* Removes broken test

* Fixes inconsistent test

This was due to Clickhouse setup not inserting the sessions with the exact same timestamp consistently and making the test inconsistent

* Combines `include` param, asyncs time_on_page and bounce_rate

* Fixes CH error when no pageviews exist in period

* Format

* Changelog

* Increases await timeout to accomodate larger data sets

* Improves handling of timeout behavior

* Fixes session-based filtering on time on page queries

* Formatting

* Removes old forced entry page modal from top pages report
6 years ago
Uku Taht d10d8fb7dd Adjust new pricing date 6 years ago
Uku Taht f16666e374 Localize billing screens 6 years ago
Uku Taht 547e888242 Localize change plan page 6 years ago
Uku Taht 88efccef2f Localize the /upgrade page 6 years ago
Uku Taht 0e481efafe Use today as change of date 6 years ago
Uku Taht 1b91909021 Grandfather old plans 6 years ago
Uku Taht 4935a62b22 Use JSON 6 years ago
Uku Taht d1ef27ec9c Make sure upgrading to unlisted plans works OK 6 years ago
Uku Taht 885b1d271a Refactor current plan data format to make changing it easier 6 years ago
Uku Taht 09df65d5ec Improve password policy 6 years ago
Uku Taht e511c9663e
Pagination for sites listing (#994)
* Add simple pagination to sites index page

* Add changelog entry
6 years ago
Uku Taht 503916980e Remove no_limit_ids 6 years ago
Uku Taht 6492b61fa5 Add test for grandfathering 6 years ago
Uku Taht c3b24342c6 Allow exemption list to be provided in config 6 years ago
Uku Taht 119b9514b2 Add limit of 20 sites 6 years ago
Uku Taht 5c759dbfed Clean up Ecto changeset code for events 6 years ago
Uku Taht f185d3736e Add hostname to user id hash 6 years ago
Uku Taht 8d1039b8ce
Upgrades Oban to v2.6.1 (#967)
* Upgrades Oban to v2.6.1

* Use correct clickhouse_ecto
6 years ago
Uku Taht b107717774
Additional metrics for timeseries API call and CSV export (#952)
* Add `metrics` option to timeseries call

* Add more metrics to csv export

* Add changelog entry
6 years ago
Uku Taht 7982c236c1 Better Sentry tracking 6 years ago
Uku Taht 6ca7425dc9
Send renewal notification for annual subscriptions (#949)
* Add background job to send renewal notification

* Use copy from Marko

* Add expiration email in case the user has cancelled

* Add queue config

* Use correct tag for expiration email
6 years ago
Uku Taht ef9ce7dbe2 Mix format 6 years ago
Uku Taht a980a57de5 Use single source of truth for plans 6 years ago
Uku Taht 264b233ca6 Fix shared link creation with password 6 years ago
Uku Taht 956e92f161 Change default timezone to Etc/UTC 6 years ago
Uku Taht 80abfa6ef6 Add validations 6 years ago
Uku Taht d473accf40 Add API key scopes 6 years ago
Uku Taht 2fc136777c Add unique index to shared link name 6 years ago
Uku Taht 3ed964b45b Add API endpoints for site and shared link creation 6 years ago
Uku Taht 43efb03081 Remove failing DB restore query 6 years ago
Uku Taht 1d892ec890 Add plans 6 years ago
Uku Taht 58cff47b6f
Add name to shared links (#910)
* Add name to shared links

* Add changelog entry
6 years ago
Uku Taht 968d280304 Fix crash when calculating visit duration with 0 sessions 6 years ago
Uku Taht 61676db4a7 Fix crash when querying bounce rate with 0 sessions 6 years ago
Uku Taht fa64a73d6e
Upgrade mailer dependency (#899)
* Upgrade mailer dependency

* Changelog
6 years ago
Uku Taht 2e16cdb15b Use UInt32 6 years ago
Uku Taht 3eb2f6d7a8
Add support for sampling (#875)
* Add support for sampling

* Use correct mix package

* Add notice about data sampling
6 years ago
Uku Taht 432ad2a606
Stats API improvements (#826)
* Show hourly data for a day

* Fix aggregations with long periods and comaprisons enabled

* Return Direct / None instead of empty string when missing source

* Fix/Align breakdown query with current queries in the UI

* Show correct number of days for 7d and 30d

* Wrap results of aggregate call in `results`

* Allow breakdown by event:name

* Ensure event:name filter works in all contexts

* WIP

* Add support for custom event props
6 years ago
Uku Taht 06177b84e6 Fix billing usage section 6 years ago
Uku Taht 4ea60f8f1c Add upgrade page for specific plan 6 years ago
Vignesh Joglekar 61c3959626
Fixes goal filters for glob goals (#781)
* Fixes goal filters for glob goals

* Format

* Consolidates goal application
6 years ago
Vignesh Joglekar b952053cee
Adds same glob patterns as exclusions for pageview goals (#750)
* First Pass - 90% wrong

* Second pass - basically done

* Swaps N+1 for multiMatchAllIndices & friends

* New version of query with just one regex computation

* Slight formatting, basic test

* Updates tests to match master new events

* Changelog

* Adds more sophisticated glob tests
6 years ago
Uku Taht c7196afa32 Add a filter for exit pages 6 years ago
Uku Taht cda031d453
Bookmarkable shared links (#752)
* Implement shared links with a static URL

* Separate sessio cookie from shared link cookie
6 years ago
Uku Taht 7fe50c3ef2 Fix check_usage 6 years ago
Uku Taht 9db24f54d5 Enable over limit notifications 6 years ago
Vignesh Joglekar ff32218bd0
Adds entry and exit pages to Top Pages module (#712)
* Initial Pass

* Adds support for page visits counting by referrer

* Includes goal selection in entry and exit computation

* Adds goal-based entry and exit page stats, formatting, code cleanup

* Changelog

* Format

* Exit rate, visit duration, updated tests

* I keep forgetting to format :/

* Tests, last time

* Fixes double counting, exit rate >100%, relevant tests

* Fixes exit pages on filter and goal states

* Adds entry and exit filters, fixes various bugs

* Fixes discussed issues

* Format

* Fixes impossible case in tests

Originally, there were only 2 pageviews for `test-site.com`,`/` on `2019-01-01`, but that doesn't make sense when there were 3 sessions that exited on the same site/date.

* Format

* Removes boolean function parameter in favor of separate function

* Adds support for queries that use `page` filter as `entry-page`

* Format

* Makes  loader/title interaction in sources report consistent
6 years ago
Uku Taht 8a455dd191
Prevent user id collisions in session tracking (#744) 6 years ago
Uku Taht 3c4931598b Fix breakdown call 6 years ago
Uku Taht a29ac44434
Api improvements (#723)
* Require date parameter with custom period

* Validate format of the `date` param with custom period

* Return proper status codes for auth failures

* Add breakdown endpoint

* Fix pagination

* Add API validations for breakdown call

* Change the breakdown endpoint to return an object instead of an array

* Remove change to aggregate call

* Change timeseries call
6 years ago
Uku Taht 9eff6b03ed Send test billing upgrade notifications 6 years ago
Uku Taht 76d8dc588c Get paddle last payment from api call 6 years ago
Uku Taht a8cb187e8d
Track billing cycles (#697)
* Add dummy check usage worker

* Add and persist last_bill_date

* Remove name clash

* Correct timing and suggestions for over limit emails

* Fix tests for trial upgrade notifications
6 years ago
Uku Taht 9453209023 Fix custom date format
Fixes #691
6 years ago
Uku Taht 5acb5b7039
Stats API (#679)
* WIP

* Add ability to filter by anything

* Add API keys

* Add version to api endpoint

* Fix API test route

* Fix API tests

* Allow 'date' parameter in '6mo' and '12mo'

* Rename session -> visit in API filters

* Filter expressions in the API

* Implement filters in aggregate call

* Add `compare` option to aggregate call

* Add way to manage API keys through the UI

* Authenticate with API key

* Use API key in tests
6 years ago
Uku Taht cb3e401dce Revert "Add Comparison Line and Improved Tooltip to Visitors Graph (#624)"
This reverts commit 979d2526d5.
6 years ago
Vignesh Joglekar 979d2526d5
Add Comparison Line and Improved Tooltip to Visitors Graph (#624)
* First pass

* Title Section of Tooltip+Formatting

* Cleanup, Formatting, Rest of Tooltip

* Fixes bug in "month" calculate_plot

Bug: where calculate_plot on "month" didn't correctly use the end date in the date range, and instead always used the current time - making requesting past chunks of 6 or 12 month data impossible

* Cleanup + Comments

* Cleanup+Percentage Formatting Changes

* Fixes Realtime Tooltip

* Prevent FE Crash

* Minor format, code cleanup

* Removes erroneous changes

* Github UI is bad at indenting
6 years ago
Uku Taht a4bf74a84c Do not allow duplicate spike notification to be created 6 years ago
Uku Taht e02377509c Add 10m, 20m plan 6 years ago
Uku Taht e7a7deef84
Clarify that custom events are billed separately from events (#607)
* Clarify that custom events are billed separately from events

* Fix for dark mode
6 years ago
Uku Taht e873d790e5
Self hosted improvements (#603)
* Introduce SELFHOST config. Enable cron by default for self hosters

* Fix cron, disable custom domain for selfhost

* Add github links

* Add notice about google search console on self hosted

* Enfore BASE_URL

* Set all selfhost users' trial expiry to 100 years from now

* Fix admin user creation
6 years ago
Uku Taht e786af648d
Fix session duration for out-of-order events (#593) 6 years ago