Make GHCi have separate flags for interactive Haskell expressions

[simonpj]

It's becoming clear that in GHCi we want to have a separate set of command-line flags for

• Compiling Haskell expressions typed at the GHCi prompt
• Compiling entire modules, eg via :load

Examples of this need are:

• We already have an ad-hoc difference in the way that type-class defaults are applied: http://www.haskell.org/ghc/docs/latest/html/users_guide/interactive-evaluation.html#extended-default-rules
• In #3202 there's a well-argued request to change behaviour of the monomorphism restriction

So the new feature would consist of:

• The InteractiveContext should contain a set of DynFlags used for compiling command-line Haskell expressions (the interactive DynFlags)
• The :set command would change both the DynFlags maintained by GHCi for compiling entire modules (the batch DynFlags), and the interactive DynFlags.
• A new :seti command to change the interactive flags
• Just possibly a :setb command to set the batch flags but leave the interactive ones unchanged.

Arguably it'd be good to have a command to display the current flag settings (of either DynFlags) but that's another blob of work.

[claus]

I'm no fan of either MonoMorphismRestriction? or GHCi's extended defaulting rules (the '()' part, in particular), but I wonder whether adding yet another level of settings would really be an improvement.

It is bad enough that GHCi doesn't easily inherit options/language flags from the modules it loads (if I work in a module that has, eg, LANGUAGE QuasiQuotes, I still need to :set -XQuasiQuotes in the GHCi session, and so on) - I wouldn't want to have to think about a third level of settings. But as long as there is a way for me never to let the various levels of settings get out of sync, I don't need to stand in the way of people who are really, really sure they want to do this to themselves..

Arguably it'd be good to have a command to display the current flag settings (of either DynFlags?) but that's another blob of work.

Part of it is there, but it is complicated by the distinction between language and other options, as well as the sheer number of options GHC has accumulated. :set not only shows current *non-language* dynamic flag settings, it has a separate group for settings that are particularly likely to be relevant for working in GHCi. And :show languages shows currently active language flags.

Prelude> :set
options currently set: none.
GHCi-specific dynamic flag settings:
  -fno-print-explicit-foralls
  -fno-print-bind-result
  -fno-break-on-exception
  -fno-break-on-error
  -fno-print-evld-with-show
other dynamic, non-language, flag settings:
  -fwarn-dodgy-foreign-imports
  -fno-warn-dodgy-imports
  -fwarn-duplicate-exports
  -fno-warn-hi-shadowing
  ...

Prelude> :show languages
active language flags:
  -XImplicitPrelude
  -XMonomorphismRestriction
  -XMonoPatBinds

I'm not exactly sure why this latter list is as it is (it was decided to show only non-default language flags there, but I've certainly not set MonomorphismRestriction explicitly)? Perhaps :show languages should be changed to show all language flag settings, with a separate group for non-default settings (or perhaps just listing for each entry whether or not it has the default setting, and use the separate group to highlight GHCi-specific flags instead, similar to :set)?

But if users are not aware of the current settings, or the ways to display/change them, are they really going to be aware of another level of settings that influences what is going on? Could raising awareness of current possibilities achieve the same goal with less confusion? Talking about confusion, I had thought the '()' part of ExtendedDefaultRules was GHCi-only (or is this distinction part of what you are aiming for with this ticket?)..

[simonpj]

The trouble is that there already is another layer of settings for interactive expressions, namely the baked-in ones concerning extended defaults. The proposal here is to make that explicit, and controllable, rather than hidden and baked in. The hidden, baked-in version is vulnerable to feature request for new baking-in, such as #3202. If it was controllable, people could do that in their .ghci file.

And if you want identical flags for both, then in your .ghci you can set the interactive flags to exactly match the batch flags.

So I think its arguable that this ticket would make things simpler. The other simplicity approach would to insist that the interactive and batch flags were identical -- but that'd mean no extended defaults which people would hate.

Simon

[claus]

1. Ok. In batch mode, in-source flags/options apply to the current module only, not to imports or importers, while commandline flags/options apply transitively to the whole session.

In GHCi, there is currently no equivalent to the former, only to the latter.

Your "interactive flags" that apply to the current session prompt only, not to code loaded during the session, would be the equivalent to GHCi prompt pragmas, just that you are suggesting a different syntax/interface.

I agree that offering both prompt and session flags would be more consistent. I'm still slightly concerned about the interface, which could easily become more confusing rather than simpler.

2. This was beginning to sound more and more like deja vu (does that even apply to sounds?-), and indeed: "Change the meaning of -fextended-default-rules" #1877

3. In current GHC head, what baked-in rules are not switched off by :set -XNoExtendedDefaults?

4. While you're dealing with this, could you please remove the current inconsistency that in-source flags are not propagated to the interactive flags? Given module

{-# LANGUAGE PatternGuards #-}

f x | () <- x = x

I would expect to have PatternGuards available when GHCi is in *Main (as opposed to Main), but that isn't the case:

Ok, modules loaded: Main.
*Main> let g x | () <- x = x

<interactive>:1:8:
    Warning: accepting non-standard pattern guards (use -XPatternGuards to suppress this message)
                 () <- x

and having to repeat all those pragma flags in the GHCi session is somewhat tedious.

[simonmar]

Replying to claus:

4. While you're dealing with this, could you please remove the current inconsistency that in-source flags are not propagated to the interactive flags? Given module {{{ {-# LANGUAGE PatternGuards? #-} f x | () <- x = x }}} I would expect to have PatternGuards available when GHCi is in *Main (as opposed to Main), but that isn't the case:

What do you want to happen when there are multiple modules in scope at the prompt? Take the union of the flags? What if they are contradictory?

[claus]

4. While you're dealing with this, could you please remove the current inconsistency that in-source flags are not propagated to the interactive flags?

What do you want to happen when there are multiple modules in scope at the prompt? Take the union of the flags? What if they are contradictory?

According to http://www.haskell.org/ghc/docs/latest/html/users_guide/interactive-evaluation.html#ghci-scope:

The syntax *module indicates that it is the full top-level scope of module that is contributing to the scope for expressions typed at the prompt. Without the *, just the exports of the module are visible. .. GHCi combines the scopes from all of these modules to form the scope that is in effect at the prompt.

Naively, I'd interpret that as meaning that the scopes are merged/unioned, and that the scopes have to be compatible for this to succeed. I'd just like this behaviour to be consistently applied to all of the module (if I were to add both Data.Map and Data.IntMap, things like empty would become ambiguous, so such contradictions aren't new).

If I'm *in* the top-level scope, the view from the prompt should be the same as from within the module. If I'm trying to combine scopes that couldn't possibly be part of a single module, I'd like to know (eg., I'd be tempted to classify :m +*Data.Map +*Data.IntMap as an error, even if the source are available, on the rationale that :m +mod means import while :m +*mod means include).

Apart from in-source pragmas, GHCi also ignores default declarations, which used to bite me when I was still using default. Given

import Data.Ratio
default (Ratio Int)

compare GHCi's

*Main> 1
1

with Hugs'

Main> 1
1 % 1

[simonmar]

Replying to claus:

(eg., I'd be tempted to classify :m +*Data.Map +*Data.IntMap as an error, even if the source are available, on the rationale that :m +mod means import while :m +*mod means include).

I'd be tempted to agree, although this is a fairly deep change. The context is currently ([Module],[Module]), this would mean changing it to Either Module [Module], or perhaps ([Module], Maybe Module). Probably this ought to be done at the same time as allowing general import declarations (#2362), since that also changes the type here.

If we made this change, then I think it makes sense to take into account the flags and defaults of the "current" module.

[claus]

(eg., I'd be tempted to classify :m +*Data.Map +*Data.IntMap as an error, even if the source are available, on the rationale that :m +mod means import while :m +*mod means include).

I'd be tempted to agree, although this is a fairly deep change. The context is currently ([Module],[Module]), this would mean changing it to Either Module [Module], or perhaps ([Module], Maybe Module).

I probably misunderstand your intentions there, but ([Module],[Module]) should be fine (assuming that this lists *-ed and other modules separately), as long as only modules with compatible pragmas, defaults, and bindings can be added to the *-ed modules list. The *Data.Map/*Data.IntMap example would fail only because both define the same names (which is possible in import/:m +mod, but not in include/:m +*mod).

If we made this change, then I think it makes sense to take into account the flags and defaults of the "current" module.

Thanks. I think I could live with only one *-ed module, if that was necessary to get this consistency, but there's no telling what wonderful uses other GHCi users have come up with, or what their preferences are.. So permitting multiple *-ed modules sounds safer - just adding the requirement that they have to be compatible.

[simonmar]

Replying to claus:

Thanks. I think I could live with only one *-ed module, if that was necessary to get this consistency, but there's no telling what wonderful uses other GHCi users have come up with, or what their preferences are.. So permitting multiple *-ed modules sounds safer - just adding the requirement that they have to be compatible.

Can you precisely define "compatibility"? Can you imagine implementing (or even describing) it in a simple way? (I can't) I have to admit, this sounds way overkill to me.

If this is really a necessary feature, can you describe a compelling use case? One that doesn't have an easy workaround?

[claus]

Can you precisely define "compatibility"? Can you imagine implementing (or even describing) it in a simple way?

Sure - the same way it is defined currently, in a single file!-) Actually, defining and implementing this would be good in any case, as there currently are a few oddities..:

• no multiple default permitted (as in a single file; this could be relaxed to permitting *identical* defaults)

• no {-# LANGUAGE X, NoX #-} (actually, that is currently permitted, but I consider it an oversight/bug..)

• no {-# OPTIONS_GHC -fthis, -fno-this #-} (again, that is currently permitted..)

If there are any non-obvious cases, simply require that all modules must have the same, identical options for those. In general, proceed as if all *-ed modules were in a single file, with allowances for identical duplicates.

My intention here is that explicit settings have priority over implicit ones (if one module doesn't have an explicit default, another does, then the combination has the explicitly given default). This merging of explicit settings might lead to a module not being loadable with the merged settings, but this isn't different from that module depending on specific settings, which should be made explicit.

An alternative would be to require identical settings in all modules to be loaded in a * combination. This would be easier to check, but almost as cumbersome as permitting only a single *-ed module.

If this is really a necessary feature, can you describe a compelling use case? One that doesn't have an easy workaround?

As I mentioned, a single *-ed module would seem to work for me. It is just that I've often been annoyed if people who couldn't imagine a use case decided that there were no use cases, so I didn't want to fall into the same trap. I don't know how many folks still read ghc-bugs, or the trac RSS feeds, so perhaps raise the issue on ghc-users, and point to this ticket?

[simonmar]

Replying to claus:

Can you precisely define "compatibility"? Can you imagine implementing (or even describing) it in a simple way?

Sure - the same way it is defined currently, in a single file!-) Actually, defining and implementing this would be good in any case, as there currently are a few oddities..: - no multiple default permitted (as in a single file; this could be relaxed to permitting *identical* defaults) - no {-# LANGUAGE X, NoX #-} (actually, that is currently permitted, but I consider it an oversight/bug..) - no {-# OPTIONS_GHC -fthis, -fno-this #-} (again, that is currently permitted..) If there are any non-obvious cases, simply require that all modules must have the same, identical options for those. In general, proceed as if all *-ed modules were in a single file, with allowances for identical duplicates. My intention here is that explicit settings have priority over implicit ones (if one module doesn't have an explicit default, another does, then the combination has the explicitly given default). This merging of explicit settings might lead to a module not being loadable with the merged settings, but this isn't different from that module depending on specific settings, which should be made explicit. An alternative would be to require identical settings in all modules to be loaded in a * combination. This would be easier to check, but almost as cumbersome as permitting only a single *-ed module.

I think you've illustrated my point quite clearly! There's no sensible way to do this that both (a) is obviously the right thing, and (b) doesn't require writing a ton of tedious and error-prone comparison code to implement.

If this is really a necessary feature, can you describe a compelling use case? One that doesn't have an easy workaround?

As I mentioned, a single *-ed module would seem to work for me. It is just that I've often been annoyed if people who couldn't imagine a use case decided that there were no use cases, so I didn't want to fall into the same trap. I don't know how many folks still read ghc-bugs, or the trac RSS feeds, so perhaps raise the issue on ghc-users, and point to this ticket?

It's more a matter of taste. A feature is much more likely to be implemented if

• it is obviously the right design. A proxy we often use for this is "can be described simply in the documentation". The best UI designs require zero documentation.
• it doesn't stretch the internal architecture, e.g. by adding new global invariants
• it has a high effort-to-benefit ratio (subjective of course, and not so applicable if someone else does the coding; hint :-)

[simonpj]

At the start of this ticket I proposed a modest change. I can't say I have followed the conversation between Claus and Simon in detail; but it seems at least that consensus has not been reached. There is a danger that the original suggestion will fall by the wayside by becoming entangled in a larger debate.

My suggestion is to implement the change I originally proposed (in brief: have a flag-set for the GHCi prompt, distinct from the ones used for compiling modules), leaving any more sophisticated feature request for another ticket.

But before doing so:

• No one except Claus has commented on the original suggestion. Does anyone case? If not, maybe it is not worth the bother?

• Claus, are you of the opinion that it'd be a retrograde step to implement the original proposal of this ticket, without implementing some version of your proposal? Or would it be a step in the right direction?

Simon

[igloo]

I agree that the original proposal makes sense.

[YitzGale]

I thought your mention of #3202 in the original description was already an implicit statement of my support for this, so I kept quiet. But yes, I do support it.

[claus]

Replying to simonpj:

..conversation between Claus and Simon in detail; but it seems at least that consensus has not been reached.

I thought consensus was fairly close:

• I'm no longer opposed to the change in principle

• I'm still slightly worried about the interface adding complexity (perhaps it would suffice to document that :set corresponds to commandline options while :seti corresponds to in-source pragmas, but it would be better to make it so)

• Simon and I seemed to agree that this separation should be followed by making the interactive options match the in-source options for the current module

• we agreed that this would be easier to achieve if there was only one *-ed module permitted per session (currently many such are permitted)

• we only disagreed on whether such a restriction might have negative effects on other use cases; Simon strongly favoured this to avoid having to define what it means for two modules to have compatible options; I tended the other way, assuming it would be useful to have this defined anyway

* Claus, are you of the opinion that it'd be a retrograde step to implement the original proposal of this ticket, without implementing some version of your proposal? Or would it be a step in the right direction?

It would be a half step in the right direction. The only disadvantage I can see is that without the other half step, the interim state might actually be more complex (three sets of options to keep track of instead of two). But as I said, I'm no longer opposed (just want more consistency and less complexity;-).

[simonpj]

I've had a chat with Simon. Here is our proposed design.

• Change GHCi so that at most one module is "fully open", currently denoted *M in GHCi's prompt. If a module M is "fully open", expressions typed at the GHCi prompt are interpreted (roughly) as if they were written in M itself. Specifically:
  • All the top-level things in the module are in scope
  • The per-module flags in {-# OPTIONS #-} and {-# LANGUAGE #-} pragmas
  • M's default declaration holds

• Note that currently more than one module can be fully open, and their top level scopes are merged. We propose to make that at most one.

• The DynFlags used to compile code will be computed as follows.
  • When compiling a module M:
    • Start with the baseline DynFlags
    • Apply flags specified on the original command-line
    • Apply flags specified by :set in this GHCi session
    • Apply flags specified in the module M itself
      
      
  • When compiling an expression typed on the GHCi command line:
    • Start with the baseline DynFlags
    • Apply flags specified on the original command-line
    • (Perhaps: apply flags specified by :set in this GHCi session. We aren't sure whether or not to do this.)
    • Apply GHCi baseline command-prompt flags (e.g. special defaulting rules)
    • Apply flags specified by :seti in this GHCi session
    • If there is a fully-open module M, apply flags specified in M itself. That is, flags in M get the last word.

• Fixing GHCi so that M's default declaration holds is a separate job, but it's really part of the same ball of wax.

Implementing this plan will require some representation changes. In particular, since we need to apply M's source-code flags in two difference places, we really need to remember the diffs with M, perhaps as a (DynFlags -> DynFlags) function or something.

This all seems quite feasible, but it's fiddly. Neither plans to implement it right away, but we'd be happy if someone else did.

Simon

[simonpj]

See also #5673 for a real-life use case.

[simonmar]

see #5778 - dealing with this is now a higher priority. Due to the fixes in #437, people who turn on extensions by default in their .ghci file will now see all their compiled modules get recompiled by GHCi because the flags have changed.

[marlowsd@…]

commit 2e55760b856540535fa0e4fe1805a75eea7d6b45

Author: Simon Marlow <marlowsd@gmail.com>
Date:   Wed Feb 29 16:23:08 2012 +0000

    GHCi: add :seti, for options that apply only at the prompt (#3217)
    
    GHCi now maintains two DynFlags: one that applies to whole modules
    loaded with :load, and one that applies to things typed at the prompt
    (expressions, statements, declarations, commands).
    
      The :set command modifies both DynFlags.  This is for backwards
      compatibility: users won't notice any difference.
    
      The :seti command applies only to the interactive DynFlags.
    
    Additionally, I made a few changes to ":set" (with no arguments):
    
      * Now it only prints out options that differ from the defaults,
        rather than the whole list.
    
      * There is a new variant, ":set -a" to print out all options (the
        old behaviour).
    
      * It also prints out language options.
    
    e.g.
    
    Prelude> :set
    options currently set: none.
    base language is: Haskell2010
    with the following modifiers:
      -XNoDatatypeContexts
      -XNondecreasingIndentation
    GHCi-specific dynamic flag settings:
    other dynamic, non-language, flag settings:
      -fimplicit-import-qualified
    warning settings:
    
    ":seti" (with no arguments) does the same as ":set", but for the
    interactive options.  It also has the "-a" option.
    
    The interactive DynFlags are kept in the InteractiveContext, and
    copied into the HscEnv at the appropriate points (all in HscMain).
    
    There are some new GHC API operations:
    
    -- | Set the 'DynFlags' used to evaluate interactive expressions.
    setInteractiveDynFlags :: GhcMonad m => DynFlags -> m ()
    
    -- | Get the 'DynFlags' used to evaluate interactive expressions.
    getInteractiveDynFlags :: GhcMonad m => m DynFlags
    
    -- | Sets the program 'DynFlags'.
    setProgramDynFlags :: GhcMonad m => DynFlags -> m [PackageId]
    
    -- | Returns the program 'DynFlags'.
    getProgramDynFlags :: GhcMonad m => m DynFlags
    
    Note I have not completed the whole of the plan outlined in #3217 yet:
    when in the context of a loaded module we don't take the interactive
    DynFlags from that module.  That needs some more refactoring and
    thinking about, because we'll need to save and restore the original
    interactive DynFlags.
    
    This solves the immediate problem that people are having with the new
    flag checking in 7.4.1, because now it is possible to set language
    options in ~/.ghci that do not affect loaded modules and thereby cause
    recompilation.

 compiler/main/DynFlags.hs        |   17 ++--
 compiler/main/GHC.hs             |   63 ++++++++++--
 compiler/main/GhcMake.hs         |   15 ++-
 compiler/main/HscMain.hs         |   52 ++++++---
 compiler/main/HscTypes.lhs       |   11 ++-
 compiler/main/InteractiveEval.hs |   17 +--
 compiler/typecheck/TcEnv.lhs     |    8 +-
 compiler/typecheck/TcSMonad.lhs  |    4 +-
 docs/users_guide/ghci.xml        |   98 +++++++++++++++++-
 ghc/GhciMonad.hs                 |    6 +-
 ghc/InteractiveUI.hs             |  218 ++++++++++++++++++++++++++------------
 ghc/Main.hs                      |    2 +
 12 files changed, 378 insertions(+), 133 deletions(-)

[simonpj]

From the commit message, here's the current status:

• Note I have not completed the whole of the plan outlined in #3217 yet: when in the context of a loaded module we don't take the interactive DynFlags from that module. That needs some more refactoring and thinking about, because we'll need to save and restore the original interactive DynFlags.

We're bumping this part to 7.6 and downgrading to 'normal'. If people care a lot, bid to upgrade!